Llm

For instructions on how to authenticate to use this endpoint, see API overview.

Create a chat completion using OpenAI or compatible models. Follows OpenAI's Chat Completions API format.

Endpoints

POST
POST

Create llm gateway v1 chat completions

Required API key scopes

task:write

Path parameters

  • project_id
    string

    Project ID of the project you're trying to access. To find the ID of the project, make a call to /api/projects/.

Query parameters

  • format
    string
    One of: "json""txt"

Request parameters

  • model
    string

    The model to use for completion (e.g., 'gpt-4', 'gpt-3.5-turbo')

  • messages
    array

    List of message objects with 'role' and 'content'

  • temperature
    number

    Sampling temperature between 0 and 2

  • top_p
    number

    Nucleus sampling parameter

  • n
    integer

    Number of completions to generate

  • stream
    boolean
    Default: false

    Whether to stream the response

  • stream_options

    Additional options for streaming

  • stop
    array

    Stop sequences

  • max_tokens
    integer

    Maximum number of tokens to generate

  • max_completion_tokens
    integer

    Maximum number of completion tokens (alternative to max_tokens)

  • presence_penalty
    number

    Presence penalty between -2.0 and 2.0

  • frequency_penalty
    number

    Frequency penalty between -2.0 and 2.0

  • logit_bias

    Logit bias mapping

  • user
    string

    Unique user identifier

  • tools
    array

    List of tools available to the model

  • tool_choice

    Controls which tool is called

  • parallel_tool_calls
    boolean

    Whether to allow parallel tool calls

  • response_format

    Format for the model output

  • seed
    integer

    Random seed for deterministic sampling

  • logprobs
    boolean

    Whether to return log probabilities

  • top_logprobs
    integer

    Number of most likely tokens to return at each position

  • modalities
    Click to open
    array

    Output modalities

  • prediction

    Prediction content for speculative decoding

  • audio

    Audio input parameters

  • reasoning_effort

    Reasoning effort level for o-series models

    • none - none
    • minimal - minimal
    • low - low
    • medium - medium
    • high - high
    • default - default
  • verbosity

    Controls the verbosity level of the model's output

    • concise - concise
    • standard - standard
    • verbose - verbose
  • store
    boolean

    Whether to store the output for model distillation or evals

  • web_search_options

    Web search tool configuration

  • functions
    array

    Deprecated in favor of tools. List of functions the model may call

  • function_call

    Deprecated in favor of tool_choice. Controls which function is called

Response


Example request

POST /api/projects/:project_id/llm_gateway/v1/chat/completions
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/llm_gateway/v1/chat/completions/\
	-d model="string",\
	-d messages="array"

Example response

Status 200 Successful response with chat completion
RESPONSE
{
  "id": "string",
  "object": "chat.completion",
  "created": 0,
  "model": "string",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "system",
        "content": "string",
        "name": "string",
        "function_call": null,
        "tool_calls": [
          null
        ]
      },
      "finish_reason": "string"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0,
    "completion_tokens_details": null,
    "prompt_tokens_details": null
  },
  "system_fingerprint": "string",
  "service_tier": "auto"
}
Status 400 Invalid request parameters
RESPONSE
{
  "error": {
    "property1": null,
    "property2": null
  }
}
Status 500 Internal server error
RESPONSE
{
  "error": {
    "property1": null,
    "property2": null
  }
}

Create llm gateway v1 messages

Create a message using Anthropic's Claude models. Compatible with Anthropic's Messages API format.

Required API key scopes

task:write

Path parameters

  • project_id
    string

    Project ID of the project you're trying to access. To find the ID of the project, make a call to /api/projects/.

Query parameters

  • format
    string
    One of: "json""txt"

Request parameters

  • model
    string

    The model to use for completion (e.g., 'claude-3-5-sonnet-20241022')

  • messages
    array

    List of message objects with 'role' and 'content'

  • max_tokens
    integer
    Default: 4096

    Maximum number of tokens to generate

  • temperature
    number

    Sampling temperature between 0 and 1

  • top_p
    number

    Nucleus sampling parameter

  • top_k
    integer

    Top-k sampling parameter

  • stream
    boolean
    Default: false

    Whether to stream the response

  • stop_sequences
    array

    Custom stop sequences

  • system

    System prompt (string or array of content blocks)

  • metadata

    Metadata to attach to the request

  • thinking

    Thinking configuration for extended thinking

  • tools
    array

    List of tools available to the model

  • tool_choice

    Controls which tool is called

  • service_tier

    Service tier for the request

    • auto - auto
    • standard_only - standard_only

Response


Example request

POST /api/projects/:project_id/llm_gateway/v1/messages
export POSTHOG_PERSONAL_API_KEY=[your personal api key]
curl 
    -H 'Content-Type: application/json'\
    -H "Authorization: Bearer $POSTHOG_PERSONAL_API_KEY" \
    <ph_app_host>/api/projects/:project_id/llm_gateway/v1/messages/\
	-d model="string",\
	-d messages="array"

Example response

Status 200 Successful response with generated message
RESPONSE
{
  "id": "string",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "property1": null,
      "property2": null
    }
  ],
  "model": "string",
  "stop_reason": "end_turn",
  "stop_sequence": "string",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "server_tool_use": null,
    "service_tier": "standard"
  }
}
Status 400 Invalid request parameters
RESPONSE
{
  "error": {
    "property1": null,
    "property2": null
  }
}
Status 500 Internal server error
RESPONSE
{
  "error": {
    "property1": null,
    "property2": null
  }
}

Community questions

Questions about this page? or post a community question.