Benchmarks resource

Access these methods through client.benchmarks.

`client.benchmarks.fetch`

Fetch detailed state for one hosted run.

Signature

async def fetch(run_id: UUID) -> SingleBenchmarkResponse

Parameters

required · UUID

Returns SingleBenchmarkResponse For examples, see the Python SDK guide.

`client.benchmarks.statuses`

Fetch lightweight status and task counts for several runs.

Signature

async def statuses(run_ids: Sequence[UUID]) -> BenchmarkStatusResponse

Parameters

required · Sequence[UUID]

Returns BenchmarkStatusResponse For examples, see the Python SDK guide.

`client.benchmarks.tasks`

Fetch a filtered and paginated page of tasks for one run.

Signature

async def tasks(run_id: UUID, request: FetchTasksRequest | None = None) -> TasksResponse

Parameters

required · UUID

optional · FetchTasksRequest | None · default:

Returns TasksResponse For examples, see the Python SDK guide.

`client.benchmarks.iter_tasks`

Iterate matching tasks from the requested offset, preserving filters and page size. Task pages use offsets; concurrent status changes can move tasks between pages.

Signature

async def iter_tasks(
    run_id: UUID,
    request: FetchTasksRequest | None = None,
) -> AsyncIterator[TaskSummary]

Parameters

required · UUID

optional · FetchTasksRequest | None · default:

Returns AsyncIterator[TaskSummary] For examples, see the Python SDK guide.

`client.benchmarks.task`

Fetch detailed state and evaluation output for one task.

Signature

async def task(run_id: UUID, task_id: str) -> SingleTaskResponse

Parameters

required · UUID

required · str

Returns SingleTaskResponse For examples, see the Python SDK guide.

`client.benchmarks.artifacts`

Fetch temporary output and log links for one task.

Signature

async def artifacts(run_id: UUID, task_id: str) -> TaskArtifactsResponse

Parameters

required · UUID

required · str

Returns TaskArtifactsResponse For examples, see the Python SDK guide.

Get started

Agents

Runs

Benchmarks

Integrations

Python SDK

CLI reference

Python SDK reference

Self-hosting

Benchmarks resource

`client.benchmarks.fetch`

`client.benchmarks.statuses`

`client.benchmarks.tasks`

`client.benchmarks.iter_tasks`

`client.benchmarks.task`

`client.benchmarks.artifacts`

​client.benchmarks.fetch

​client.benchmarks.statuses

​client.benchmarks.tasks

​client.benchmarks.iter_tasks

​client.benchmarks.task

​client.benchmarks.artifacts

`client.benchmarks.fetch`

`client.benchmarks.statuses`

`client.benchmarks.tasks`

`client.benchmarks.iter_tasks`

`client.benchmarks.task`

`client.benchmarks.artifacts`