Agent contract reference

contract.yaml sits at the top of an agent’s directory. To write one step by step, see Package an agent.

Fields

Field	Required	Meaning
`name`	yes	The agent’s name in your library, unless you upload it with `--name`. Letters, digits, `.`, `_`, and `-`.
`install_cmd`	yes	Shell command that installs the agent. `""` skips installation.
`run_cmd`	yes	Shell command that runs the agent on one task. Must contain `{problem_statement_path}`.
`final_output`	no	Absolute path of the file or directory to keep after the run.
`output_artifacts`	no	Up to 10 more files to keep individually.
`secrets`	no	Environment variables to fill from organization secrets.
`defaults`	no	The `model` parameter, which `--model` sets.
`kwargs`	no	Other parameters, which `-k` sets.
`install_egress`	no	Network access while installing.
`turns`	no	Commands for benchmarks that run the agent several times.
`ingest_lambda`	no	The Lambda that `valkyrie run analyze` sends the run to.
`provided`	no	Notes for people reading the contract. Valkyrie ignores it.

Valkyrie ignores keys it doesn’t recognize, so a misspelled field has no effect.

Order of a task

For each task, Valkyrie:

Copies the agent directory into the sandbox at /bundle/AGENT_NAME/, where AGENT_NAME is the agent’s name in your library.
Runs install_cmd from that directory, before or after the benchmark sets the task up, as the benchmark chooses.
Runs run_cmd from the benchmark’s working directory, with placeholders filled in and secrets set as environment variables.
Saves final_output and output_artifacts, even when the agent fails.
Has the benchmark grade the sandbox.

The `install_cmd` field

Runs with sh -c in /bundle/AGENT_NAME/, with a 10-minute limit.
A failed installation is tried up to three more times, then once more in a new sandbox, before the task fails.
Uploading leaves out .git, .venv, venv, and __pycache__ directories, and every file whose name contains .env, .git, venv, .pyc, .pyo, .pyd, .so, .dll, .dylib, .egg-info, or .DS_Store. Symbolic links are refused.

Placeholders

Valkyrie fills these in run_cmd:

Placeholder	Value
`{problem_statement_path}`	Path of the file holding the task’s problem statement. Required.
`{task_id}`	The task’s ID, such as `astropy__astropy-12907`.
`{model}` and other parameter names	The parameter’s value
`{slot_dir}`, `{container_name}`	See `turns`.

Values go in as they are, without shell quoting, so quote any that can contain spaces. A placeholder with no value, such as a parameter with no default that the run didn’t set, stays in the command as written.

Parameters

Parameters are values a run passes to the agent. --model sets the model parameter, declared under defaults. -k NAME VALUE sets parameters declared under kwargs.

Key	Required	Meaning
`type`	yes	`str`, `int`, `float`, `bool`, or `dict`
`required`	yes	Whether every run must set it
`default`	no	The value when a run doesn’t set it
`description`	no	What it’s for
`choices`	no	The only values a run can pass, as strings

A run that leaves out a required parameter fails to start, naming the parameter.
-k with a name the contract doesn’t declare is ignored.
A dict parameter can’t be set with -k, so give it a default.

Secrets

secrets maps each environment variable to a secret reference that valkyrie secret set printed:

contract.yaml

secrets:
  ANTHROPIC_API_KEY: anthropic-api-key

Valkyrie reads each secret when it creates the task’s sandbox. Secret values are never saved with the run.
-s ENV_VAR SECRET on valkyrie run start adds or replaces a mapping for that run.
When a secret’s value is a JSON object, each variable gets the key with its own name, so one secret can hold several keys.

The `install_egress` field

What the agent can reach while it installs:

Value	Effect
`"*"`	Allows all outbound traffic. This is the default.
`[]`	Blocks all outbound traffic.
A list	Allows only those hostnames, URLs, IPv4 addresses, and CIDR ranges.

IPv6 addresses aren’t supported. On Daytona, one list can’t mix hostnames with CIDR ranges.
The benchmark sets what the agent can reach while it runs, and what setup and grading can reach.
egress_allowlist, an earlier field that some contracts still set, adds hosts to the benchmark’s run allowlist, or becomes the run allowlist when the benchmark sets none.
Hosts behind a shared content delivery network (CDN) can make an allowlist broader than the names in it.

Output files

final_output is saved as agent_output.tar.gz in the task’s folder, if the path exists when the agent finishes. While the task is active, its files can also be read live from the sandbox, including any written during grading, so don’t treat the path as private. output_artifacts entries take one of two forms:

Form	Example	Reads from
A path	`artifacts/summary.json`	`/tmp/valkyrie/artifacts/summary.json`
An object	`{path: artifacts/result.json, source: /logs/{task_id}/result.json}`	`source`

source is an absolute path. It can include {task_id} and a glob under a directory, such as /logs/*.json. A glob saves its first match in sorted order.
Every entry is required unless it sets required: false. A required artifact that is missing or can’t be saved fails the task. An optional one is skipped and logged.
A task can declare at most 10 artifacts, of at most 250 MiB each and 250 MiB together.
Entries are saved under benchmarks/RUN_ID/TASK_ID/ at their path, where RUN_ID and TASK_ID are the run’s and task’s IDs.

Error reports

When run_cmd exits with a nonzero code, the task fails with Agent command failed with exit code N. A line written to the file named by VALKYRIE_ERROR_PATH is added to that error.

Write one line in the form Type: message, such as AgentError: model returned no patch.
A report longer than 2048 bytes is dropped.
Valkyrie replaces the values of the contract’s secrets with [REDACTED]. It can’t recognize other sensitive data, so don’t write prompts or model output.
Exit code 137, when the operating system stops the agent, isn’t an agent failure. The benchmark still grades the task.

The time an agent can spend on a task is set by the benchmark. See Task time limits.

The `turns` field

Some benchmarks start the agent more than once per task, for example to continue a conversation, and can run several copies, called slots, side by side. Other benchmarks ignore turns.

contract.yaml

run_cmd: agent --input "{problem_statement_path}" --home "{slot_dir}"
turns:
  continue_cmd: agent resume --input "{problem_statement_path}" --home "{slot_dir}"
  interrupt_grace_seconds: 90

Key	Meaning
`continue_cmd`	Command for a slot’s later turns. `run_cmd` starts its first turn.
`interrupt_grace_seconds`	Seconds a stopped turn gets after an interrupt signal before Valkyrie stops it. Without it, Valkyrie stops the turn at once.
`container_name`	Name of the container a turn starts, with `{slot}` filled in. Not allowed when slots run side by side.

{problem_statement_path} is each turn’s input.
{slot_dir} is a directory private to the slot. When slots run side by side, run_cmd must use {slot_dir}, and so must final_output if it’s set.
{container_name} is the value of container_name for the current slot.

Get started

Agents

Runs

Benchmarks

Integrations

Python SDK

CLI reference

Python SDK reference

Self-hosting

Agent contract reference

Fields

Order of a task

The `install_cmd` field

Placeholders

Parameters

Secrets

The `install_egress` field

Output files

Error reports

The `turns` field

​Fields

​Order of a task

​The install_cmd field

​Placeholders

​Parameters

​Secrets

​The install_egress field

​Output files

​Error reports

​The turns field

Fields

Order of a task

The `install_cmd` field

Placeholders

Parameters

Secrets

The `install_egress` field

Output files

Error reports

The `turns` field