AI observability
For instructions on how to authenticate to use this endpoint, see API overview.
Endpoints
GET | |
PATCH | |
DELETE | |
GET | |
POST | |
GET | |
GET | |
GET | |
GET | |
POST | |
GET | |
GET | |
GET | |
GET | |
GET | |
POST | |
GET | |
GET | |
GET | |
GET |
Retrieve ai observability offline experiments results payload
Required API key scopes
evaluation:readllm_analytics:readPath parameters
- idstring
- result_idstring
Response
Example request
GET /api /projects /:project_id /ai_observability /offline_experiments /:id /results /:result_id /payloadExample response
Status 200
Status 400
Status 401
Status 403
Status 404
Status 429
Status 503
Update ai observability offline experiments results payload
Required API key scopes
evaluation:writellm_analytics:readPath parameters
- idstring
- result_idstring
Request parameters
- revisioninteger
- payload
Response
Example request
PATCH /api /projects /:project_id /ai_observability /offline_experiments /:id /results /:result_id /payloadExample response
Status 200
Status 400
Status 401
Status 403
Status 404
Status 409
Status 429
Status 503
Delete ai observability offline experiments results payload
Required API key scopes
evaluation:writellm_analytics:readPath parameters
- idstring
- result_idstring
Query parameters
- revisioninteger
Example request
DELETE /api /projects /:project_id /ai_observability /offline_experiments /:id /results /:result_id /payloadExample response
Status 204 No response body
Status 400
Status 401
Status 403
Status 404
Status 409
Status 429
Status 503
List all ai observability offline experiments scorer summaries
Required API key scopes
evaluation:readllm_analytics:readPath parameters
- idstring
Query parameters
- cursorstring
- limitintegerDefault:
50 - scorer_definition_idstring
- scorer_version_idsstring
Response
Example request
GET /api /projects /:project_id /ai_observability /offline_experiments /:id /scorer_summariesExample response
Status 200
Status 400
Status 401
Status 403
Status 404
Status 429
Status 503
Create ai observability offline experiments upload
Required API key scopes
offline_evaluation_ingestion:writePath parameters
- idstring
Request parameters
- itemsarray
- resultsarray
Response
Example request
POST /api /projects /:project_id /ai_observability /offline_experiments /:id /uploadExample response
Status 200
Status 400
Status 401
Status 402
Status 403
Status 404
Status 409
Status 413
Status 429
List all ai observability offline scorers history
Required API key scopes
evaluation:readllm_analytics:readPath parameters
- idstring
Query parameters
- application_versionstring
- breakdownstringOne of:
"model_version""prompt_version""application_version""suite_key""run_source" - created_byinteger
- created_by_is_nullboolean
- cursorstring
- dataset_identifierstring
- dataset_revision_identifierstring
- dataset_sourcestring
- date_fromstring
- date_tostring
- limitintegerDefault:
50 - model_versionstring
- prompt_versionstring
- run_sourcestring
- scorer_version_idsstring
- searchstring
- statusesstring
- suite_keystring
- tag_matchstringDefault:
anyOne of:"any""all" - tagsstring
Response
Example request
GET /api /projects /:project_id /ai_observability /offline_scorers /:id /historyExample response
Status 200
Status 400
Status 401
Status 403
Status 404
Status 429
Status 503
List all ai observability traces
The project's traces in a date range, newest first, a page at a time. The first page also carries how many traces the selection holds.
Required API key scopes
llm_analytics:readQuery parameters
- cursorstring
- date_fromstring
- date_tostring
- filter_test_accountsboolean | null
- group_keystring
- group_type_indexinteger
- limitintegerDefault:
50 - person_idstring
- propertiesstring
- searchstring
- viewstringOne of:
"errors""feedback"
Response
Example request
GET /api /projects /:project_id /ai_observability /tracesExample response
Status 200
Status 400 A parameter, the cursor, the date range or a property filter is invalid.
Status 403 The key lacks a required scope, or a property filter needs the query:read scope.
Status 429 Too many queries are running for the project or organization.
Retrieve ai observability traces clusters
The clusters that list this trace, from the newest scheduled run of each clustering job that lists it.
Required API key scopes
llm_analytics:readai_observability_clusters:readPath parameters
- idstring
Query parameters
- created_atstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /clustersExample response
Status 200
Status 400 The created_at parameter or the encoded trace id is malformed.
Status 429 Too many queries are running for the project or organization.
Retrieve ai observability traces evaluations
A trace's evaluation results, newest first and at most 250, each with its outcome and label ready to render, plus a roll-up of the latest run of each evaluation on each node.
Required API key scopes
llm_analytics:readPath parameters
- idstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /evaluationsExample response
Status 200
Status 400 The encoded trace id is malformed.
Status 429 Too many queries are running for the project or organization.
Create ai observability traces exports
Starts writing the trace and every node, inputs and outputs included, to a JSON file. Poll the export until it completes, then download it.
Required API key scopes
llm_analytics:readPath parameters
- idstring
Request parameters
- timestamp_hintstring
Response
Example request
POST /api /projects /:project_id /ai_observability /traces /:id /exportsExample response
Status 201
Status 400 The timestamp hint or the encoded trace id is malformed.
Status 429 Too many exports were requested.
Retrieve ai observability traces exports
The state of one of your exports of this trace.
Required API key scopes
llm_analytics:readPath parameters
- export_idinteger
- idstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /exports /:export_idExample response
Status 200
Status 400 The encoded trace id is malformed.
Status 404 You have no export with this id for this trace.
Retrieve ai observability traces exports content
Downloads a completed export as a JSON file.
Required API key scopes
llm_analytics:readPath parameters
- export_idinteger
- idstring
Query parameters
- purposestringDefault:
downloadOne of:"download""copy"
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /exports /:export_id /contentExample response
Status 200
Status 400 The encoded trace id or the purpose is malformed.
Status 404 You have no export with this id for this trace.
Status 409 The export is still being written, or failed.
Retrieve ai observability traces feedback
End-user feedback on a trace: scores and comments from its feedback events, survey responses linked to it, and surveys shown but not answered.
Required API key scopes
llm_analytics:readPath parameters
- idstring
Query parameters
- timestamp_hintstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /feedbackExample response
Status 200
Status 400 The timestamp hint or the encoded trace id is malformed.
Status 429 Too many queries are running for the project or organization.
Retrieve ai observability traces nodes
One node of a trace: its input and output ready to render, its event name, uuid and timestamp, its error and its metadata. The raw event properties are on the raw read.
Required API key scopes
llm_analytics:readPath parameters
- idstring
- node_idstring
Query parameters
- timestamp_hintstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /nodes /:node_idExample response
Status 200
Status 400 The timestamp hint or an encoded id is malformed.
Status 404 No node with this id in the trace.
Status 429 Too many queries are running for the project or organization.
Retrieve ai observability traces nodes dataset item
One node of a trace as a dataset item: its input and output as ingested, and its custom properties as metadata. The root node gives the trace's input and output state with no metadata.
Required API key scopes
llm_analytics:readPath parameters
- idstring
- node_idstring
Query parameters
- timestamp_hintstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /nodes /:node_id /dataset_itemExample response
Status 200
Status 400 The timestamp hint is missing or malformed, or an encoded id is malformed.
Status 404 No node with this id in the trace.
Status 429 Too many queries are running for the project or organization.
Create ai observability traces nodes parser test
Tests a custom parser recipe against one node without saving it: the node's input and output as they would render with the parser saved, and whether it parses every side that the project's saved parsers leave unparsed.
Required API key scopes
llm_analytics:readPath parameters
- idstring
- node_idstring
Query parameters
- timestamp_hintstring
Request parameters
- sourcestring
Response
Example request
POST /api /projects /:project_id /ai_observability /traces /:id /nodes /:node_id /parser_testExample response
Status 200
Status 400 The parser does not compile, with the reason in `detail`, or the timestamp hint or an encoded id is malformed.
Status 404 No node with this id in the trace.
Status 429 Too many queries are running for the project or organization.
Retrieve ai observability traces nodes raw
One node of a trace as ingested: its event properties and the keys that hold its input, output and tools, with nothing normalized.
Required API key scopes
llm_analytics:readPath parameters
- idstring
- node_idstring
Query parameters
- timestamp_hintstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /nodes /:node_id /rawExample response
Status 200
Status 400 The timestamp hint or an encoded id is malformed.
Status 404 No node with this id in the trace.
Status 429 Too many queries are running for the project or organization.
Retrieve ai observability traces pull request
The pull request the trace's git branch belongs to, from engineering analytics.
Required API key scopes
llm_analytics:readengineering_analytics:readPath parameters
- idstring
Query parameters
- timestamp_hintstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /pull_requestExample response
Status 200
Status 400 The timestamp hint or the encoded trace id is malformed.
Status 403 The engineering-analytics feature flag is off for the project, or the API key lacks a scope.
Status 429 Too many queries are running for the project or organization.
Retrieve ai observability traces review
The trace's human review with its scores ready to render, and the review queue the trace waits in.
Required API key scopes
llm_analytics:readPath parameters
- idstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /reviewExample response
Status 200
Status 400 The encoded trace id is malformed.
Retrieve ai observability traces search
Find text in a trace's steps: each matching step in tree order with its number of occurrences.
Required API key scopes
llm_analytics:readPath parameters
- idstring
Query parameters
- qstring
- timestamp_hintstring
Response
Example request
GET /api /projects /:project_id /ai_observability /traces /:id /search




