AI Observability
Contents
AI Observability captures every call your AI product makes to an LLM – the prompt, the response, the tokens, the cost, the latency, and the tools it reached for along the way – and stitches them into traces you can actually read. When something goes wrong, you see the whole conversation instead of a single failed request.
Because it runs on the same events as the rest of PostHog, every trace comes with the user who triggered it, their session replay, and any exceptions the request threw. That same trace data is what evaluations score and what Self-driving reads when it files a report about your AI product's quality.
Where you can use it
You explore traces and debug your AI product in the PostHog web app. The other surfaces let you investigate the same data from your editor, from your own code, and from the Self-driving reports it produces.
PostHog Web
Read traces and sessions, watch cost and latency, cluster failures, and test prompts in the playground.
Explore traces →PostHog MCP
Pull up traces, costs, and errors from any MCP client or AI editor while you're still in the code.
Query traces →PostHog Desktop Beta
Read the Self-driving investigation reports your trace data produces, and decide what to do about them.
Review findings →API
Send traces in, query them back out, and drive evaluations, reviews, and clustering from your own code.
Use the API →Where its data comes from
AI Observability runs on the LLM events your app sends to PostHog. Most of them come from an SDK wrapping your model provider, but you can also forward traces from another observability tool or attach feedback from your users.
Traces and spans
Generations, spans, and traces captured by the PostHog SDKs across every model provider and framework.
Install an SDK →Integrations
Forward traces you already collect in Helicone, Langfuse, Traceloop, or Keywords AI.
Connect a tool →User feedback
Thumbs up and down, ratings, and comments attached to the exact trace they're about.
Collect feedback →How AI Observability works with Self-driving
Your LLM traces are a signal source for Self-driving. A scout watches cost, latency, errors, volume, and evaluation performance, and when a slice regresses against its own baseline, it files an investigation report in the inbox.
A cost step or a latency band widening is rarely a one-line code fix – it's a question about a model, a prompt, or a change in what your users are asking for. So this scout stops at the report and routes it to a human reviewer. It does not open a draft pull request.
You read traces in the PostHog web app, and read the reports in the Self-driving inbox or PostHog Desktop. Other signal sources, like error tracking, do open pull requests – see the Self-driving docs for the full picture.