Evaluations

For instructions on how to authenticate to use this endpoint, see API overview.

Endpoints

GET
POST
GET
PATCH
DELETE
POST

List all evaluations

Required API key scopes

evaluation:read

Path parameters

  • project_id
    string

Query parameters

  • enabled
    boolean
  • id__in
    array
  • limit
    integer
  • offset
    integer
  • order_by
    array
  • search
    string

Response


Example request

GET /api/environments/:project_id/evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:project_id/evaluations/

Example response

Status 200
RESPONSE
{
  "count": 123,
  "next": "http://api.example.org/accounts/?offset=400&limit=100",
  "previous": "http://api.example.org/accounts/?offset=200&limit=100",
  "results": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "name": "string",
      "description": "string",
      "enabled": true,
      "status": "active",
      "status_reason": "trial_limit_reached",
      "evaluation_type": "llm_judge",
      "evaluation_config": null,
      "output_type": "boolean",
      "output_config": null,
      "conditions": null,
      "model_configuration": {
        "provider": "openai",
        "model": "string",
        "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
        "provider_key_name": "string"
      },
      "created_at": "2019-08-24T14:15:22Z",
      "updated_at": "2019-08-24T14:15:22Z",
      "created_by": {
        "id": 0,
        "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
        "distinct_id": "string",
        "first_name": "string",
        "last_name": "string",
        "email": "user@example.com",
        "is_email_verified": true,
        "hedgehog_config": {
          "property1": null,
          "property2": null
        },
        "role_at_organization": "engineering"
      },
      "deleted": true
    }
  ]
}

Create evaluations

Required API key scopes

evaluation:write

Path parameters

  • project_id
    string

Request parameters

  • name
    string
  • description
    string
  • enabled
    boolean
  • evaluation_type
  • evaluation_config
  • output_type
  • output_config
  • conditions
  • model_configuration
  • deleted
    boolean

Response


Example request

POST /api/environments/:project_id/evaluations
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:project_id/evaluations/\
	-d name="string",\
	-d evaluation_type=undefined,\
	-d output_type=undefined

Example response

Status 201
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": null,
  "output_type": "boolean",
  "output_config": null,
  "conditions": null,
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {
      "property1": null,
      "property2": null
    },
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Retrieve evaluations

Required API key scopes

evaluation:read

Path parameters

  • id
    string
  • project_id
    string

Response


Example request

GET /api/environments/:project_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:project_id/evaluations/:id/

Example response

Status 200
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": null,
  "output_type": "boolean",
  "output_config": null,
  "conditions": null,
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {
      "property1": null,
      "property2": null
    },
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Update evaluations

Required API key scopes

evaluation:write

Path parameters

  • id
    string
  • project_id
    string

Request parameters

  • name
    string
  • description
    string
  • enabled
    boolean
  • evaluation_type
  • evaluation_config
  • output_type
  • output_config
  • conditions
  • model_configuration
  • deleted
    boolean

Response


Example request

PATCH /api/environments/:project_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl -X PATCH \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:project_id/evaluations/:id/\
	-d name="string"

Example response

Status 200
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": null,
  "output_type": "boolean",
  "output_config": null,
  "conditions": null,
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {
      "property1": null,
      "property2": null
    },
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Delete evaluations

Hard delete of this model is not allowed. Use a patch API call to set "deleted" to true

Required API key scopes

evaluation:write

Path parameters

  • id
    string
  • project_id
    string

Example request

DELETE /api/environments/:project_id/evaluations/:id
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl  -X DELETE \
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:project_id/evaluations/:id/

Example response

Status 405 No response body

Create evaluations test hog

Test Hog evaluation code against sample events without saving.

Required API key scopes

evaluation:read

Path parameters

  • project_id
    string

Request parameters

  • name
    string
  • description
    string
  • enabled
    boolean
  • evaluation_type
  • evaluation_config
  • output_type
  • output_config
  • conditions
  • model_configuration
  • deleted
    boolean

Response


Example request

POST /api/environments/:project_id/evaluations/test_hog
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/environments/:project_id/evaluations/test_hog/\
	-d name="string",\
	-d evaluation_type=undefined,\
	-d output_type=undefined

Example response

Status 200
RESPONSE
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "name": "string",
  "description": "string",
  "enabled": true,
  "status": "active",
  "status_reason": "trial_limit_reached",
  "evaluation_type": "llm_judge",
  "evaluation_config": null,
  "output_type": "boolean",
  "output_config": null,
  "conditions": null,
  "model_configuration": {
    "provider": "openai",
    "model": "string",
    "provider_key_id": "f265db88-9bcc-4e5b-add5-bfd9a815465c",
    "provider_key_name": "string"
  },
  "created_at": "2019-08-24T14:15:22Z",
  "updated_at": "2019-08-24T14:15:22Z",
  "created_by": {
    "id": 0,
    "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f",
    "distinct_id": "string",
    "first_name": "string",
    "last_name": "string",
    "email": "user@example.com",
    "is_email_verified": true,
    "hedgehog_config": {
      "property1": null,
      "property2": null
    },
    "role_at_organization": "engineering"
  },
  "deleted": true
}

Community questions

Questions about this page? or post a community question.