Llm

For instructions on how to authenticate to use this endpoint, see API overview.

Create a chat completion using OpenAI or compatible models. Follows OpenAI's Chat Completions API format.

Endpoints

POST
POST

Create llm gateway v1 chat completions

Required API key scopes

task:write

Path parameters

  • project_id
    string

    Project ID of the project you're trying to access. To find the ID of the project, make a call to /api/projects/.


Query parameters

  • format
    string
    One of: "json""txt"

Request parameters

  • model
    string

    The model to use for completion (e.g., 'gpt-4', 'gpt-3.5-turbo')

  • messages
    array

    List of message objects with 'role' and 'content'

  • temperature
    number

    Sampling temperature between 0 and 2

  • top_p
    number

    Nucleus sampling parameter

  • n
    integer

    Number of completions to generate

  • stream
    boolean
    Default: false

    Whether to stream the response

  • stream_options

    Additional options for streaming

  • stop
    array

    Stop sequences

  • max_tokens
    integer

    Maximum number of tokens to generate

  • max_completion_tokens
    integer

    Maximum number of completion tokens (alternative to max_tokens)

  • presence_penalty
    number

    Presence penalty between -2.0 and 2.0

  • frequency_penalty
    number

    Frequency penalty between -2.0 and 2.0

  • logit_bias

    Logit bias mapping

  • user
    string

    Unique user identifier

  • tools
    array

    List of tools available to the model

  • tool_choice

    Controls which tool is called

  • parallel_tool_calls
    boolean

    Whether to allow parallel tool calls

  • response_format

    Format for the model output

  • seed
    integer

    Random seed for deterministic sampling

  • logprobs
    boolean

    Whether to return log probabilities

  • top_logprobs
    integer

    Number of most likely tokens to return at each position

  • modalities
    Click to open
    array

    Output modalities

  • prediction

    Prediction content for speculative decoding

  • audio

    Audio input parameters

  • reasoning_effort

    Reasoning effort level for o-series models

    • none - none
    • minimal - minimal
    • low - low
    • medium - medium
    • high - high
    • default - default
  • verbosity

    Controls the verbosity level of the model's output

    • concise - concise
    • standard - standard
    • verbose - verbose
  • store
    boolean

    Whether to store the output for model distillation or evals

  • web_search_options

    Web search tool configuration

  • functions
    array

    Deprecated in favor of tools. List of functions the model may call

  • function_call

    Deprecated in favor of tool_choice. Controls which function is called


Response


Example request

POST /api/projects/:project_id/llm_gateway/v1/chat/completions
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_gateway/v1/chat/completions/\
-d model="string",\
-d messages="array"

Example response

Status 200 Successful response with chat completion
RESPONSE
{
"id": "string",
"object": "chat.completion",
"created": 0,
"model": "string",
"choices": [
{
"index": 0,
"message": {
"role": "system",
"content": "string",
"name": "string",
"function_call": null,
"tool_calls": [
null
]
},
"finish_reason": "string"
}
],
"usage": {
"prompt_tokens": 0,
"completion_tokens": 0,
"total_tokens": 0,
"completion_tokens_details": null,
"prompt_tokens_details": null
},
"system_fingerprint": "string",
"service_tier": "auto"
}
Status 400 Invalid request parameters
RESPONSE
{
"error": {
"property1": null,
"property2": null
}
}
Status 500 Internal server error
RESPONSE
{
"error": {
"property1": null,
"property2": null
}
}

Create llm gateway v1 messages

Create a message using Anthropic's Claude models. Compatible with Anthropic's Messages API format.

Required API key scopes

task:write

Path parameters

  • project_id
    string

    Project ID of the project you're trying to access. To find the ID of the project, make a call to /api/projects/.


Query parameters

  • format
    string
    One of: "json""txt"

Request parameters

  • model
    string

    The model to use for completion (e.g., 'claude-3-5-sonnet-20241022')

  • messages
    array

    List of message objects with 'role' and 'content'

  • max_tokens
    integer
    Default: 4096

    Maximum number of tokens to generate

  • temperature
    number

    Sampling temperature between 0 and 1

  • top_p
    number

    Nucleus sampling parameter

  • top_k
    integer

    Top-k sampling parameter

  • stream
    boolean
    Default: false

    Whether to stream the response

  • stop_sequences
    array

    Custom stop sequences

  • system

    System prompt (string or array of content blocks)

  • metadata

    Metadata to attach to the request

  • thinking

    Thinking configuration for extended thinking

  • tools
    array

    List of tools available to the model

  • tool_choice

    Controls which tool is called

  • service_tier

    Service tier for the request

    • auto - auto
    • standard_only - standard_only

Response


Example request

POST /api/projects/:project_id/llm_gateway/v1/messages
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl
-H 'Content-Type: application/json'\
-H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
<ph_app_host>/api/projects/:project_id/llm_gateway/v1/messages/\
-d model="string",\
-d messages="array"

Example response

Status 200 Successful response with generated message
RESPONSE
{
"id": "string",
"type": "message",
"role": "assistant",
"content": [
{
"property1": null,
"property2": null
}
],
"model": "string",
"stop_reason": "end_turn",
"stop_sequence": "string",
"usage": {
"input_tokens": 0,
"output_tokens": 0,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"server_tool_use": null,
"service_tier": "standard"
}
}
Status 400 Invalid request parameters
RESPONSE
{
"error": {
"property1": null,
"property2": null
}
}
Status 500 Internal server error
RESPONSE
{
"error": {
"property1": null,
"property2": null
}
}

Community questions

Questions about this page? or post a community question.