Resume pending work with an optional concurrency override.
Full syntax for each command below is in the run update, run stop, run resume, and run retry reference, and in client.runs.resume and client.runs.retry for Python.
Update concurrency
The tracker applies the persisted limit when it next refreshes the run. Increasing it admits more tasks. Decreasing it does not cancel in-flight work and pauses new admissions until usage falls below the limit.
The value must be positive. Completed, stopped, and failed runs cannot be updated. The limit is enforced independently by each process_benchmark executor, so overlapping executors can temporarily exceed the persisted value.
Stop work
Stop a whole run, or name the tasks to stop:
A plain stop does not interrupt work that already started. --force also deletes the sandboxes of in-progress and evaluating tasks and marks those tasks stopped:
Force-stopping deletes running sandboxes and cannot be undone. The CLI asks for confirmation first.
Choose resume or retry
resume admits work that never ran. retry re-runs tasks that ended in ERROR:
Either accepts a task subset, a concurrency override, and --connect to stream afterwards. Three overrides change what the retried work uses:
--update-agent refreshes the frozen agent from its current S3 bundle, so a re-pushed agent takes effect.--from-scratch clears stored evaluation resume state, so a task is graded again from the beginning.--benchmark-url replaces the benchmark-service URL stored on the run, which a tunnelled service needs after its URL changes.
-s merges a secret mapping into the stored contract, which is how a rotated credential reaches resumed tasks.
For a terminal run, a selected valid task ID without an existing row is created as new PENDING work. An in-progress retry rejects selected IDs that do not already exist in ERROR state.
Export benchmark task IDs
The output path is optional. Pass the file to --task-ids-file on start, stop, resume, or retry.