Skip to main content
Start a run with an agent from your library and a benchmark, in a project you created once:
Valkyrie prints the run ID, which every other valkyrie run command takes. Every option is in the valkyrie run start reference. To start runs from Python, see Manage runs with Python.

​
Pass the model and parameters

--model sets the agent’s model parameter, and -k NAME VALUE sets the others. The agent contract says which parameters exist and which are required.

​
Run part of a dataset

A run covers the benchmark’s default dataset unless you choose otherwise. Pick another dataset with --dataset:
Run a slice of the dataset for a cheap first try:
To pick tasks by ID, export the dataset’s task IDs, edit the file, and pass it back:
--task-ids takes a comma-separated list instead.

​
Set concurrency

--concurrency sets how many tasks run at once, 5 by default. A higher value finishes sooner but spends sandbox and model budget faster, and can reach your model provider’s rate limits. You can change it while the run goes.

​
Pass secrets and headers

-s ENV_VAR SECRET sets an environment variable in the agent’s sandbox from one of your organization’s secrets, and wins over the contract’s mapping for the same variable. It takes the secret’s name, never its value:
-H NAME VALUE sends an extra header to the benchmark service. See Benchmark authentication.

​
Choose the sandbox provider

--provider picks the provider for this run. Without it, the run uses your default provider, or daytona. See Configure sandbox providers.

​
Use the hosted benchmark service

When you have saved your own URL for a benchmark, your runs use it. --ignore-custom-services, or --ics, sends this run to Vals AI’s hosted service instead, and keeps your saved URL.

​
Set a priority

On a Tracker that queues tasks in a shared sandbox pool, --priority orders this run’s tasks against other waiting work, from 0, the highest, to 4, the lowest. Runs without a priority get 3.
Where the selected provider isn’t queued, --priority fails with Queue priority requires ...; leave it out. valkyrie queue status shows the queue.

​
Label a run

The label is saved on the run. Find every run with it later with valkyrie run list -l sweep-a.

​
Put the run in a project

Every CLI start names a project by its handle with --project. The handle is lowercase letters, digits and hyphens, such as google-report-xyz, and it never changes. The name, such as Google Report XYZ, is the label people read. The project holds the spend type, subcategory, customer and extra tags, so you set them once when you create it. Valkyrie saves the project with the run, so failed and stopped runs keep it, and cost reports can group spend by it.
--name, --spend-type and --subcategory are required. --tag key=value adds extra tags and can be repeated. List your organization’s projects:
To move a run to another project later, in any run state, retag it. Retagging adds a new link and keeps the earlier ones, and valkyrie run fetch shows the newest:

​
Watch the run as it starts

--connect streams progress until the run finishes. Without it, attach later with valkyrie run fetch RUN_ID --connect, where RUN_ID is the ID that run start printed. See Monitor runs.

​
Start several runs at once

--count starts up to 10 identical, independent runs, one after another. The Valkyrie CLI stops at the first start that fails, so if one fails, check valkyrie run list before you start the rest again. --connect can’t be combined with --count.
Each run uses its own concurrency, so --count 3 --concurrency 20 can run 60 tasks at once.

​
Limits

  • If starting, retrying, or resuming a run fails with This user cannot start or retry benchmarks, starting runs is turned off for your account. You can still read and stop runs. Ask Vals AI to turn it back on.