Invoke a Lambda after a run

Valkyrie can call an AWS Lambda function after a run finishes and its results are saved, for example to send a notification or start your own analysis. Valkyrie calls the function with the Tracker’s own AWS credentials, so the function must be in the Tracker’s AWS account or allow that account to invoke it. A local Tracker can’t call Lambda functions.

Write the handler

lambda_function.py

def lambda_handler(event, context):
    run_id = event["benchmark_id"]
    agent_name = event["contract"]["name"]

    # Your post-run work, such as a notification

    return {"statusCode": 200, "body": f"Processed {agent_name} run {run_id}"}

Name the function when you start a run

valkyrie run start --agent my-agent --benchmark swebench --lambda my-post-run-handler --project my-project

To call another function when a rerun finishes, pass --lambda to valkyrie run resume or valkyrie run retry.

When the function runs

Valkyrie checks that it can invoke the function when the run starts, and the run fails if it can’t.
The function runs once, after every task has finished, and only if the run ends as FINISHED. A stopped run doesn’t call it.
Resuming or retrying the run calls it again when the rerun finishes.
A function that fails, by raising an exception or returning a statusCode of 400 or more, doesn’t change the run’s status. To confirm that the function ran, check its own logs.

Payload

The function receives the run’s launch settings, plus its ID and benchmark name. These are the fields most handlers need:

{
  "benchmark_id": "e532551e-d51b-4912-983d-47695bd24174",
  "benchmark_name": "swebench",
  "dataset": "default",
  "concurrency": 5,
  "task_ids": ["astropy__astropy-12907"],
  "slice_str": null,
  "lambda_function": "my-post-run-handler",
  "contract": {
    "name": "my-agent",
    "install_cmd": "bash setup.sh",
    "run_cmd": "my-agent --task \"{problem_statement_path}\"",
    "final_output": "/logs/my-agent",
    "secrets": {"ANTHROPIC_API_KEY": "anthropic-api-key"}
  }
}

Field	Meaning
`benchmark_id`	The run’s ID
`benchmark_name`	The benchmark the run used
`dataset`	The dataset the run used
`concurrency`	The run’s concurrency
`task_ids`	The tasks the run was limited to, or `null` for all
`slice_str`	The run’s `--slice`, or `null`
`lambda_function`	This function’s name
`contract`	The agent contract the run used, with secret names but not their values

Get started

Agents

Runs

Benchmarks

Integrations

Python SDK

CLI reference

Python SDK reference

Self-hosting

Invoke a Lambda after a run

Write the handler

Name the function when you start a run

When the function runs

Payload

​Write the handler

​Name the function when you start a run

​When the function runs

​Payload

Write the handler

Name the function when you start a run

When the function runs

Payload