Skip to main content
valkyrie run analyze hands a finished run to an AWS Lambda function, the analyzer, that turns the agent’s outputs into Transluce Docent records and returns a link to read them. Valkyrie saves the link on the run. The analyzer decides what to upload, how to summarize it, and which models to use. Valkyrie invokes the analyzer the same way as a post-run Lambda, so the same AWS account requirement applies, and a local Tracker can’t run analyses.

​
Write an analyzer

Each agent writes its outputs in its own shape, so each agent needs its own analyzer. Valkyrie calls it with:
The analyzer needs to do the following:
  1. Read the run’s outputs from s3://S3_BUCKET/benchmarks/BENCHMARK_ID/, where S3_BUCKET and BENCHMARK_ID are the payload’s s3_bucket and benchmark_id. Each task’s folder holds its agent_output.tar.gz and any output artifacts the contract declared.
  2. Turn each task into a Docent record and upload it. See the Docent documentation for the record format.
  3. Return a JSON object with the reading link in reading_plan_url:
Other returned fields are ignored. A response without reading_plan_url marks the analysis DONE with no link, and raising an exception marks it ERROR. The analyzer needs read access to the run’s folder in the bucket.

​
Name the analyzer in the contract

Add ingest_lambda to the agent’s contract.yaml and upload the agent again:
contract.yaml
valkyrie run analyze reads ingest_lambda from the agent’s current version in your library, not from the contract the run used. Adding it and uploading again makes earlier runs of that agent analyzable too.

​
Analyze a run

Replace RUN_ID with the run’s ID. The run must be FINISHED. The command streams progress and prints the reading link when the analyzer returns. valkyrie run fetch RUN_ID also shows the link when it exists. A run that already has a link returns it without calling the analyzer again. After you fix the analyzer, run it again with --no-cache:
A failed analysis isn’t retried, because a second call can upload the run twice. An analysis can run for up to 15 minutes, the Lambda limit.

​
Analysis statuses

valkyrie run fetch RUN_ID shows the run’s analysis status: