Evaluations

For instructions on how to authenticate to use this endpoint, see API overview.

Endpoints

GET
POST
GET
PATCH
DELETE
POST
GET
POST
GET
PATCH
DELETE
POST

List all environments evaluations

Required API key scopes

evaluation:read

Query parameters

  • enabled
    boolean
  • id__in
    array
  • limit
    integer
  • offset
    integer
  • order_by
    array
  • search
    string

Response


Example request

GET /api/environments/:environment_id/evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:environment_id/evaluations/

Example response

Status 200
RESPONSE
{
  "count": 123,
  "next": "http://api.example.org/accounts/?offset=400&limit=100",
  "previous": "http://api.example.org/accounts/?offset=200&limit=100",
  "results": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "name": "string",
      "description": "string",
      "enabled": true,
      "status": "active",
      "status_reason": "trial_limit_reached",
      "evaluation_type": "llm_judge",
      "evaluation_config": {
        "prompt": "string"
      },
      "output_type": "boolean",
      "output_config": {
        "allows_na": false
      },
      "conditions": [
        {
          "id": "string",
          "rollout_percentage": 100,
          "properties": [
            {}
          ]
        }
      ],
      "model_configuration": {
        "provider": "openai",
        "model": "string",
        "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
        "provider_key_name": "string"
      },
      "created_at": "2019-08-24T14:15:22Z",
      "updated_at": "2019-08-24T14:15:22Z",
      "created_by": {
        "id": 0,
        "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
        "distinct_id": "string",
        "first_name": "string",
        "last_name": "string",
        "email": "user@example.com",
        "is_email_verified": true,
        "hedgehog_config": {},
        "role_at_organization": "engineering"
      },
      "deleted": true
    }
  ]
}

Create environments evaluations

Required API key scopes

evaluation:write

Request parameters

  • name
    string
  • description
    string
  • enabled
    boolean
  • evaluation_type
  • evaluation_config
  • output_type
  • output_config
    object
  • conditions
    Click to open
    array
  • model_configuration
  • deleted
    boolean

Response


Example request

POST /api/environments/:environment_id/evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:environment_id/evaluations/\
	-d name="string",\
	-d evaluation_type=undefined,\
	-d output_type=undefined

Example response

Status 201
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": {
    "prompt": "string"
  },
  "output_type": "boolean",
  "output_config": {
    "allows_na": false
  },
  "conditions": [
    {
      "id": "string",
      "rollout_percentage": 100,
      "properties": [
        {}
      ]
    }
  ],
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Retrieve environments evaluations

Required API key scopes

evaluation:read

Path parameters

  • id
    string

Response


Example request

GET /api/environments/:environment_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:environment_id/evaluations/:id/

Example response

Status 200
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": {
    "prompt": "string"
  },
  "output_type": "boolean",
  "output_config": {
    "allows_na": false
  },
  "conditions": [
    {
      "id": "string",
      "rollout_percentage": 100,
      "properties": [
        {}
      ]
    }
  ],
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Update environments evaluations

Required API key scopes

evaluation:write

Path parameters

  • id
    string

Request parameters

  • name
    string
  • description
    string
  • enabled
    boolean
  • evaluation_type
  • evaluation_config
  • output_type
  • output_config
    object
  • conditions
    Click to open
    array
  • model_configuration
  • deleted
    boolean

Response


Example request

PATCH /api/environments/:environment_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:environment_id/evaluations/:id/\
	-d name="string"

Example response

Status 200
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": {
    "prompt": "string"
  },
  "output_type": "boolean",
  "output_config": {
    "allows_na": false
  },
  "conditions": [
    {
      "id": "string",
      "rollout_percentage": 100,
      "properties": [
        {}
      ]
    }
  ],
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Delete environments evaluations

Hard delete of this model is not allowed. Use a patch API call to set "deleted" to true

Required API key scopes

evaluation:write

Path parameters

  • id
    string

Example request

DELETE /api/environments/:environment_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl  -X DELETE \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:environment_id/evaluations/:id/

Example response

Status 405 No response body

Create environments evaluations test hog

Test Hog evaluation code against sample events without saving.

Required API key scopes

evaluation:read

Request parameters

  • source
    string
  • sample_count
    integer
    Default: 5
  • allows_na
    boolean
    Default: false
  • conditions
    array

Response


Example request

POST /api/environments/:environment_id/evaluations/test_hog
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:environment_id/evaluations/test_hog/\
	-d source="string"

Example response

Status 200
RESPONSE
{
  "results": [
    {
      "event_uuid": "string",
      "trace_id": "string",
      "input_preview": "string",
      "output_preview": "string",
      "result": true,
      "reasoning": "string",
      "error": "string"
    }
  ],
  "message": "string"
}

List all evaluations

Required API key scopes

evaluation:read

Query parameters

  • enabled
    boolean
  • id__in
    array
  • limit
    integer
  • offset
    integer
  • order_by
    array
  • search
    string

Response


Example request

GET /api/projects/:project_id/evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/evaluations/

Example response

Status 200
RESPONSE
{
  "count": 123,
  "next": "http://api.example.org/accounts/?offset=400&limit=100",
  "previous": "http://api.example.org/accounts/?offset=200&limit=100",
  "results": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "name": "string",
      "description": "string",
      "enabled": true,
      "status": "active",
      "status_reason": "trial_limit_reached",
      "evaluation_type": "llm_judge",
      "evaluation_config": {
        "prompt": "string"
      },
      "output_type": "boolean",
      "output_config": {
        "allows_na": false
      },
      "conditions": [
        {
          "id": "string",
          "rollout_percentage": 100,
          "properties": [
            {}
          ]
        }
      ],
      "model_configuration": {
        "provider": "openai",
        "model": "string",
        "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
        "provider_key_name": "string"
      },
      "created_at": "2019-08-24T14:15:22Z",
      "updated_at": "2019-08-24T14:15:22Z",
      "created_by": {
        "id": 0,
        "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
        "distinct_id": "string",
        "first_name": "string",
        "last_name": "string",
        "email": "user@example.com",
        "is_email_verified": true,
        "hedgehog_config": {},
        "role_at_organization": "engineering"
      },
      "deleted": true
    }
  ]
}

Create evaluations

Required API key scopes

evaluation:write

Request parameters

  • name
    string
  • description
    string
  • enabled
    boolean
  • evaluation_type
  • evaluation_config
  • output_type
  • output_config
    object
  • conditions
    Click to open
    array
  • model_configuration
  • deleted
    boolean

Response


Example request

POST /api/projects/:project_id/evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/evaluations/\
	-d name="string",\
	-d evaluation_type=undefined,\
	-d output_type=undefined

Example response

Status 201
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": {
    "prompt": "string"
  },
  "output_type": "boolean",
  "output_config": {
    "allows_na": false
  },
  "conditions": [
    {
      "id": "string",
      "rollout_percentage": 100,
      "properties": [
        {}
      ]
    }
  ],
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Retrieve evaluations

Required API key scopes

evaluation:read

Path parameters

  • id
    string

Response


Example request

GET /api/projects/:project_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/evaluations/:id/

Example response

Status 200
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": {
    "prompt": "string"
  },
  "output_type": "boolean",
  "output_config": {
    "allows_na": false
  },
  "conditions": [
    {
      "id": "string",
      "rollout_percentage": 100,
      "properties": [
        {}
      ]
    }
  ],
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Update evaluations

Required API key scopes

evaluation:write

Path parameters

  • id
    string

Request parameters

  • name
    string
  • description
    string
  • enabled
    boolean
  • evaluation_type
  • evaluation_config
  • output_type
  • output_config
    object
  • conditions
    Click to open
    array
  • model_configuration
  • deleted
    boolean

Response


Example request

PATCH /api/projects/:project_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/evaluations/:id/\
	-d name="string"

Example response

Status 200
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": {
    "prompt": "string"
  },
  "output_type": "boolean",
  "output_config": {
    "allows_na": false
  },
  "conditions": [
    {
      "id": "string",
      "rollout_percentage": 100,
      "properties": [
        {}
      ]
    }
  ],
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {},
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Delete evaluations

Hard delete of this model is not allowed. Use a patch API call to set "deleted" to true

Required API key scopes

evaluation:write

Path parameters

  • id
    string

Example request

DELETE /api/projects/:project_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl  -X DELETE \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/evaluations/:id/

Example response

Status 405 No response body

Create evaluations test hog

Test Hog evaluation code against sample events without saving.

Required API key scopes

evaluation:read

Request parameters

  • source
    string
  • sample_count
    integer
    Default: 5
  • allows_na
    boolean
    Default: false
  • conditions
    array

Response


Example request

POST /api/projects/:project_id/evaluations/test_hog
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/evaluations/test_hog/\
	-d source="string"

Example response

Status 200
RESPONSE
{
  "results": [
    {
      "event_uuid": "string",
      "trace_id": "string",
      "input_preview": "string",
      "output_preview": "string",
      "result": true,
      "reasoning": "string",
      "error": "string"
    }
  ],
  "message": "string"
}

Community questions

Questions about this page? or post a community question.