Skip to main content
The Valkyrie CLI and SDK need credentials and a Tracker to send requests to. A login covers both on your own machine, and an access key covers both in continuous integration (CI).

​
Sign in

Your browser opens to sign in with your Google account. On a machine without a browser, add --no-browser and open the printed link yourself. The login is saved for this Tracker under ~/.config/valkyrie/sessions, readable only by you, and refreshes itself. When it can’t, the CLI tells you to run valkyrie login again.

​
Join an organization

Runs, agents, secrets, and access keys belong to an organization, and its members see its runs. Vals AI creates organizations: to get one, send an email to contact@vals.ai. To join an existing one:
  1. Ask one of its admins to run valkyrie org invite you@example.com with your email address.
  2. Sign in with valkyrie login by using that address.
  3. List your invitations with valkyrie org list.
  4. Accept one with valkyrie org accept acme.
Invitations expire after 14 days. Decline one with valkyrie org decline acme.

​
Switch organizations

valkyrie org list shows your organizations and marks the one your requests act for. Switch with:

​
Manage members

Only admins can invite and remove people:
Removing a member also deletes the access keys they created for the organization. Organization commands always use your login, even when VALKYRIE_API_KEY is set.

​
Create an access key for CI

CI jobs and machines without a browser use an access key instead of a login. While logged in, create one:
The key is printed once, on standard output, and its ID and expiry on standard error. Set the key as VALKYRIE_API_KEY where the job runs. Keys last from 1 to 90 days, 30 by default. There is no command to list keys, so keep the key ID. Revoke a key with:
Replace KEY_ID with the key ID that valkyrie token create printed. A revoked key can keep working for up to an hour. Creating and revoking keys needs a login, not an access key. Runs started with a key use its creator’s default provider and saved benchmark services, so they change when the creator changes those. To pin a CI run, pass --provider and --ignore-custom-services to valkyrie run start.

​
Choose a Tracker

The CLI and SDK use Vals AI’s hosted Tracker, https://benchmark-tracker.vals.ai, until you sign in to another one. To use your own Tracker, or another URL Vals AI gave you, sign in with VALKYRIE_URL:
Later commands use the Tracker you last signed in to. Logins are saved per Tracker, so you can stay signed in to several and pick one for a single command with VALKYRIE_URL. valkyrie logout forgets the current Tracker’s login, and commands go back to the hosted Tracker. Requests go to the first Tracker set by:
  1. url in the SDK’s ValkyrieConfig
  2. VALKYRIE_URL
  3. Your last valkyrie login, when no access key is set in api_key or VALKYRIE_API_KEY
  4. https://benchmark-tracker.vals.ai
A Tracker you run yourself without authentication needs no login or key. See Run Tracker locally.

​
Configure the Python SDK

ValkyrieClient() uses the same Tracker and login as the CLI. To choose them in code, pass a ValkyrieConfig:
Replace ACCESS_KEY with an access key, or leave out api_key to use your login. Requests authenticate with api_key, then VALKYRIE_API_KEY, then your saved login for that Tracker. Every field is in the ValkyrieConfig reference.

​
Environment variables