Benchmark services resource

Access these methods through client.services.

`client.services.catalog`

List hosted benchmark services visible to the configured tenant.

Signature

async def catalog() -> BenchmarkServiceCatalogResponse

Returns BenchmarkServiceCatalogResponse For examples, see the Python SDK guide.

`client.services.check`

Health-check services through Tracker, such as custom URLs alongside catalog entries.

Signature

async def check(services: Sequence[BenchmarkServiceEntry]) -> BenchmarkServicesResponse

Parameters

required · Sequence[BenchmarkServiceEntry]

Returns BenchmarkServicesResponse For examples, see the Python SDK guide.

`client.services.list`

List hosted services and return their current health.

Signature

async def list() -> BenchmarkServicesResponse

Returns BenchmarkServicesResponse For examples, see the Python SDK guide.

`client.services.task_ids`

Fetch task IDs for a benchmark dataset.

Signature

async def task_ids(
    benchmark: str,
    *,
    dataset: str | None = None,
    service_headers: Mapping[str, str] | None = None,
    ignore_custom_services: bool = False,
) -> list[str]

Parameters

required · str

optional · str | None · default:

optional · Mapping[str, str] | None · default:

optional · bool · default:

Returns list[str] For examples, see the Python SDK guide.

Get started

Agents

Runs

Benchmarks

Integrations

Python SDK

CLI reference

Python SDK reference

Self-hosting

Benchmark services resource

`client.services.catalog`

`client.services.check`

`client.services.list`

`client.services.task_ids`

​client.services.catalog

​client.services.check

​client.services.list

​client.services.task_ids

`client.services.catalog`

`client.services.check`

`client.services.list`

`client.services.task_ids`