Llm

For instructions on how to authenticate to use this endpoint, see API overview.

Endpoints

GET
POST
GET
PATCH
DELETE
POST
GET
POST
GET
PATCH
DELETE
POST
GET
POST
GET
GET
POST
GET
PATCH

List all llm analytics parser recipes

Required API key scopes

llm_analytics:read

Query parameters

  • limit
    integer
  • offset
    integer

Response


Example request

GET /api/projects/:project_id/llm_analytics/parser_recipes
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/parser_recipes/

Example response

Status 200
RESPONSE
{
"count": 123,
"next": "http://api.example.org/accounts/?offset=400&limit=100",
"previous": "http://api.example.org/accounts/?offset=200&limit=100",
"results": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"source": "string",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z"
}
]
}

Create llm analytics parser recipes

Required API key scopes

llm_analytics:write

Request parameters

  • name
    string
  • source
    string

Response


Example request

POST /api/projects/:project_id/llm_analytics/parser_recipes
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/parser_recipes/\
-d name="string",\
-d source="string"

Example response

Status 201
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"source": "string",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z"
}

Retrieve llm analytics parser recipes

Required API key scopes

llm_analytics:read

Path parameters

  • id
    string

Response


Example request

GET /api/projects/:project_id/llm_analytics/parser_recipes/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/parser_recipes/:id/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"source": "string",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z"
}

Update llm analytics parser recipes

Required API key scopes

llm_analytics:write

Path parameters

  • id
    string

Request parameters

  • name
    string
  • source
    string

Response


Example request

PATCH /api/projects/:project_id/llm_analytics/parser_recipes/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/parser_recipes/:id/\
-d name="string"

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"source": "string",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z"
}

Delete llm analytics parser recipes

Required API key scopes

llm_analytics:write

Path parameters

  • id
    string

Example request

DELETE /api/projects/:project_id/llm_analytics/parser_recipes/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X DELETE \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/parser_recipes/:id/

Example response

Status 204 No response body

Create llm analytics provider key validations

Validate LLM provider API keys without persisting them

Required API key scopes

llm_provider_key:write

Example request

POST /api/projects/:project_id/llm_analytics/provider_key_validations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_key_validations/

Example response

Status 200

List all llm analytics provider keys

Required API key scopes

llm_provider_key:read

Query parameters

  • limit
    integer
  • offset
    integer

Response


Example request

GET /api/projects/:project_id/llm_analytics/provider_keys
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/

Example response

Status 200
RESPONSE
{
"count": 123,
"next": "http://api.example.org/accounts/?offset=400&limit=100",
"previous": "http://api.example.org/accounts/?offset=200&limit=100",
"results": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}
]
}

Create llm analytics provider keys

Required API key scopes

llm_provider_key:write

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

POST /api/projects/:project_id/llm_analytics/provider_keys
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/\
-d provider=undefined,\
-d name="string"

Example response

Status 201
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Retrieve llm analytics provider keys

Required API key scopes

llm_provider_key:read

Path parameters

  • id
    string

Response


Example request

GET /api/projects/:project_id/llm_analytics/provider_keys/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/:id/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Update llm analytics provider keys

Required API key scopes

llm_provider_key:write

Path parameters

  • id
    string

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

PATCH /api/projects/:project_id/llm_analytics/provider_keys/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/:id/\
-d provider=undefined

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Delete llm analytics provider keys

Required API key scopes

llm_provider_key:write

Path parameters

  • id
    string

Example request

DELETE /api/projects/:project_id/llm_analytics/provider_keys/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X DELETE \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/:id/

Example response

Status 204 No response body

Create llm analytics provider keys assign

Assign this key to evaluations and optionally re-enable them.

Path parameters

  • id
    string

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

POST /api/projects/:project_id/llm_analytics/provider_keys/:id/assign
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/:id/assign/\
-d provider=undefined,\
-d name="string"

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Retrieve llm analytics provider keys dependent configs

Get evaluations using this key and alternative keys for replacement.

Path parameters

  • id
    string

Response


Example request

GET /api/projects/:project_id/llm_analytics/provider_keys/:id/dependent_configs
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/:id/dependent_configs/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Create llm analytics provider keys validate

Path parameters

  • id
    string

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

POST /api/projects/:project_id/llm_analytics/provider_keys/:id/validate
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/:id/validate/\
-d provider=undefined,\
-d name="string"

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Retrieve llm analytics provider keys trial evaluations

List enabled evaluations currently using trial credits for a given provider.

Response


Example request

GET /api/projects/:project_id/llm_analytics/provider_keys/trial_evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/provider_keys/trial_evaluations/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

List all llm analytics review queue items

Required API key scopes

llm_analytics:read

Query parameters

  • limit
    integer
  • offset
    integer
  • order_by
    string
  • queue_id
    string
  • search
    string
  • trace_id
    string
  • trace_id__in
    string

Response


Example request

GET /api/projects/:project_id/llm_analytics/review_queue_items
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/review_queue_items/

Example response

Status 200
RESPONSE
{
"count": 123,
"next": "http://api.example.org/accounts/?offset=400&limit=100",
"previous": "http://api.example.org/accounts/?offset=200&limit=100",
"results": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}
]
}

Create llm analytics review queue items

Required API key scopes

llm_analytics:write

Request parameters

  • queue_id
    string
  • trace_id
    string

Response


Example request

POST /api/projects/:project_id/llm_analytics/review_queue_items
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/review_queue_items/\
-d queue_id="string",\
-d trace_id="string"

Example response

Status 201
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Retrieve llm analytics review queue items

Required API key scopes

llm_analytics:read

Path parameters

  • id
    string

Response


Example request

GET /api/projects/:project_id/llm_analytics/review_queue_items/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/review_queue_items/:id/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Update llm analytics review queue items

Required API key scopes

llm_analytics:write

Path parameters

  • id
    string

Request parameters

  • queue_id
    string

Response


Example request

PATCH /api/projects/:project_id/llm_analytics/review_queue_items/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_analytics/review_queue_items/:id/\
-d queue_id="string"

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Community questions

Questions about this page? or post a community question.