Declare the analyzer Lambda
Addingest_lambda to the agent’s contract.yaml and push the agent:
ingest_lambda from the agent’s current pushed contract, not from the contract stored on the run. Adding the field and re-pushing makes earlier runs of that agent analyzable.
Analyze a run
FINISHED. In-progress, stopping, stopped, and errored runs are rejected with a message naming the current status. See the run analyze reference for full syntax.
The tracker invokes the Lambda with the run’s stored AWS credentials and streams progress until it returns. On success the command prints the reading URL.
Re-run the analysis after fixing the Lambda:
--no-cache, a run that already has a stored URL returns that URL without invoking the Lambda again.
Where the result is stored
The reading URL and its status are stored on the run, sovalkyrie run fetch <run-id> shows the URL once it exists.
An invocation can run for up to 15 minutes, the AWS Lambda ceiling. Valkyrie does not retry a failed invocation, because a second invocation would ingest the run twice.
Write an analyzer Lambda
Each agent writes its outputs in its own shape, so each agent needs its own analyzer Lambda. Valkyrie invokes it with this payload:- Read the run’s task outputs from
s3://<s3_bucket>/benchmarks/<benchmark_id>/. Each task directory holds that task’sagent_output.tar.gzand any output artifacts the contract declared. - Convert each task into a Docent record and upload it.
- Return a JSON object containing
reading_plan_urlso Valkyrie can store and display the reading.
reading_plan_url marks the analysis DONE without storing a URL. Raising an exception marks the run’s analysis ERROR.
Grant the Lambda read access to the run’s S3 prefix, and give the credentials used by Valkyrie lambda:InvokeFunction on it. See AWS permissions.
For Docent’s own record format and API, see the Docent documentation.