Llm

For instructions on how to authenticate to use this endpoint, see API overview.

Endpoints

POST
POST
GET
POST
GET
PATCH
DELETE
POST
GET
POST
GET
GET
POST
GET
PATCH
DELETE
GET
POST
GET

Create environments llm analytics offline evaluations experiment items

Required API key scopes

llm_analytics:read

Request parameters

  • experiment_id
    string
  • date_from
    stringnull
  • date_to
    stringnull

Response


Example request

POST /api/environments/:environment_id/llm_analytics/offline_evaluations/experiment_items
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/offline_evaluations/experiment_items/\
-d experiment_id="string"

Example response

Status 200
RESPONSE
{
"results": [
[
null
]
]
}
Status 400
Status 500

Create environments llm analytics provider key validations

Validate LLM provider API keys without persisting them

Required API key scopes

llm_provider_key:write

Example request

POST /api/environments/:environment_id/llm_analytics/provider_key_validations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_key_validations/

Example response

Status 200

List all environments llm analytics provider keys

Required API key scopes

llm_provider_key:read

Query parameters

  • limit
    integer
  • offset
    integer

Response


Example request

GET /api/environments/:environment_id/llm_analytics/provider_keys
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/

Example response

Status 200
RESPONSE
{
"count": 123,
"next": "http://api.example.org/accounts/?offset=400&limit=100",
"previous": "http://api.example.org/accounts/?offset=200&limit=100",
"results": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}
]
}

Create environments llm analytics provider keys

Required API key scopes

llm_provider_key:write

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

POST /api/environments/:environment_id/llm_analytics/provider_keys
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/\
-d provider=undefined,\
-d name="string"

Example response

Status 201
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Retrieve environments llm analytics provider keys

Required API key scopes

llm_provider_key:read

Path parameters

  • id
    string

Response


Example request

GET /api/environments/:environment_id/llm_analytics/provider_keys/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/:id/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Update environments llm analytics provider keys

Required API key scopes

llm_provider_key:write

Path parameters

  • id
    string

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

PATCH /api/environments/:environment_id/llm_analytics/provider_keys/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/:id/\
-d provider=undefined

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Delete environments llm analytics provider keys

Required API key scopes

llm_provider_key:write

Path parameters

  • id
    string

Example request

DELETE /api/environments/:environment_id/llm_analytics/provider_keys/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X DELETE \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/:id/

Example response

Status 204 No response body

Create environments llm analytics provider keys assign

Assign this key to evaluations and optionally re-enable them.

Path parameters

  • id
    string

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

POST /api/environments/:environment_id/llm_analytics/provider_keys/:id/assign
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/:id/assign/\
-d provider=undefined,\
-d name="string"

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Retrieve environments llm analytics provider keys dependent configs

Get evaluations using this key and alternative keys for replacement.

Path parameters

  • id
    string

Response


Example request

GET /api/environments/:environment_id/llm_analytics/provider_keys/:id/dependent_configs
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/:id/dependent_configs/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Create environments llm analytics provider keys validate

Path parameters

  • id
    string

Request parameters

  • provider
  • name
    string
  • api_key
    string
  • azure_endpoint
    string
  • api_version
    string
  • set_as_active
    boolean
    Default: false

Response


Example request

POST /api/environments/:environment_id/llm_analytics/provider_keys/:id/validate
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/:id/validate/\
-d provider=undefined,\
-d name="string"

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

Retrieve environments llm analytics provider keys trial evaluations

List enabled evaluations currently using trial credits for a given provider.

Response


Example request

GET /api/environments/:environment_id/llm_analytics/provider_keys/trial_evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/provider_keys/trial_evaluations/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"provider": "openai",
"name": "string",
"state": "unknown",
"error_message": "string",
"api_key": "string",
"api_key_masked": "string",
"azure_endpoint": "http://example.com",
"api_version": "string",
"azure_endpoint_display": "string",
"api_version_display": "string",
"set_as_active": false,
"created_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"last_used_at": "2019-08-24T14:15:22Z"
}

List all environments llm analytics review queue items

Required API key scopes

llm_analytics:read

Query parameters

  • limit
    integer
  • offset
    integer
  • order_by
    string
  • queue_id
    string
  • search
    string
  • trace_id
    string
  • trace_id__in
    string

Response


Example request

GET /api/environments/:environment_id/llm_analytics/review_queue_items
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queue_items/

Example response

Status 200
RESPONSE
{
"count": 123,
"next": "http://api.example.org/accounts/?offset=400&limit=100",
"previous": "http://api.example.org/accounts/?offset=200&limit=100",
"results": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}
]
}

Create environments llm analytics review queue items

Required API key scopes

llm_analytics:write

Request parameters

  • queue_id
    string
  • trace_id
    string

Response


Example request

POST /api/environments/:environment_id/llm_analytics/review_queue_items
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queue_items/\
-d queue_id="string",\
-d trace_id="string"

Example response

Status 201
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Retrieve environments llm analytics review queue items

Required API key scopes

llm_analytics:read

Path parameters

  • id
    string

Response


Example request

GET /api/environments/:environment_id/llm_analytics/review_queue_items/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queue_items/:id/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Update environments llm analytics review queue items

Required API key scopes

llm_analytics:write

Path parameters

  • id
    string

Request parameters

  • queue_id
    string

Response


Example request

PATCH /api/environments/:environment_id/llm_analytics/review_queue_items/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queue_items/:id/\
-d queue_id="string"

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"queue_id": "cefd6192-7a66-4699-a2fc-dbb7f43ad507",
"queue_name": "string",
"trace_id": "string",
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Delete environments llm analytics review queue items

Required API key scopes

llm_analytics:write

Path parameters

  • id
    string

Example request

DELETE /api/environments/:environment_id/llm_analytics/review_queue_items/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X DELETE \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queue_items/:id/

Example response

Status 204 No response body

List all environments llm analytics review queues

Required API key scopes

llm_analytics:read

Query parameters

  • limit
    integer
  • name
    string
  • offset
    integer
  • order_by
    string
  • search
    string

Response


Example request

GET /api/environments/:environment_id/llm_analytics/review_queues
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queues/

Example response

Status 200
RESPONSE
{
"count": 123,
"next": "http://api.example.org/accounts/?offset=400&limit=100",
"previous": "http://api.example.org/accounts/?offset=200&limit=100",
"results": [
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"pending_item_count": 0,
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}
]
}

Create environments llm analytics review queues

Required API key scopes

llm_analytics:write

Request parameters

  • name
    string

Response


Example request

POST /api/environments/:environment_id/llm_analytics/review_queues
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queues/\
-d name="string"

Example response

Status 201
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"pending_item_count": 0,
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Retrieve environments llm analytics review queues

Required API key scopes

llm_analytics:read

Path parameters

  • id
    string

Response


Example request

GET /api/environments/:environment_id/llm_analytics/review_queues/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/environments/:environment_id/llm_analytics/review_queues/:id/

Example response

Status 200
RESPONSE
{
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
"name": "string",
"pending_item_count": 0,
"created_at": "2019-08-24T14:15:22Z",
"updated_at": "2019-08-24T14:15:22Z",
"created_by": {
"id": 0,
"uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
"distinct_id": "string",
"first_name": "string",
"last_name": "string",
"email": "user@example.com",
"is_email_verified": true,
"hedgehog_config": {},
"role_at_organization": "engineering"
},
"team": 0
}

Community questions

Questions about this page? or post a community question.