> AI agents: this is one page from PostHog's docs. Full index of Markdown docs for LLMs: https://posthog.com/llms.txt

# Self-driving with AI Observability

**Open beta**

Self-driving is in open beta. It's improving quickly – expect rough edges, and expect them to disappear fast.

To power [Self-driving](/docs/self-driving.md), AI Observability turns your LLM data into [signals](/docs/self-driving/signals.md): agents investigate what changed, and reports land in your [inbox](/docs/self-driving/inbox.md) – where one click turns a finding into a pull request.

All of it needs LLM data flowing first – if you haven't instrumented yet, start with [installation](/docs/ai-observability/installation.md).

## How it works

A [signal](/docs/self-driving/signals.md) is a structured finding: what's happening, with the evidence behind it. Your LLM data produces signals three ways:

| Signal | What it catches | When it runs |
| --- | --- | --- |
| [Eval reports](#eval-reports) | Quality and sentiment trends in production traffic | After each batch of evaluation results |
| [Anomaly investigations](#anomaly-investigations) (alpha) | Spikes in cost, latency, or errors | When an anomaly alert fires |
| [Scouts](#scouts) | Regressions across cost, latency, errors, volume, and eval performance – or anything you describe | On a schedule you set |

## Eval reports

[Evaluations](/docs/ai-evals.md) score your live generations in production – with an LLM judge, deterministic Hog code, or sentiment analysis. Eval reports are what make those scores actionable: instead of you watching a pass-rate chart, an agent reviews each batch of results and summarizes what it found, with example generations as evidence.

To set them up:

1.  [Create an evaluation](/docs/ai-evals.md#choosing-an-evaluation-type). PostHog provisions a report configuration automatically that you can further configure and have sent to email or Slack.
2.  Enable the **AI observability** source in your [inbox](https://app.posthog.com/inbox) settings, so report findings flow into your inbox as signals.

## Anomaly investigations

**Anomaly investigations are in alpha**

Anomaly investigations are currently in alpha, as part of the **product analytics** signal source. We'd love to [hear your feedback](https://app.posthog.com/ai-observability#panel=support%3Afeedback%3Allm-analytics%3Alow%3Atrue) as we develop this feature.

Every LLM event behaves like a standard PostHog event, so cost, latency, and error rate are insights you can put an [anomaly alert](/docs/alerts.md#anomaly-detection) on. When one fires, an investigation agent picks it up before the notification reaches you: it digs into the underlying traces, writes up what it found, and suppresses the alert when the spike turns out to be a false positive – a batch job, a provider incident, one developer testing locally. You only read true positives.

To set them up:

1.  Create an insight on the metric you care about – cost, latency, or errors. The [AI Observability dashboard](/docs/ai-observability/dashboard.md) charts are a good starting point.
2.  Add an [anomaly detection alert](/docs/alerts.md#creating-an-anomaly-detection-alert) to it – the detectors learn what normal looks like from your data. Fixed-threshold alerts fire like regular alerts, without an investigation.
3.  Enable the **product analytics** source in your [inbox](https://app.posthog.com/inbox) settings, so investigation findings land there as signals.

## Scouts

[Scouts](/docs/self-driving/scouts.md) are scheduled agents that explore your LLM data and file a report only when they find something noteworthy. They can be used to catch regressions across cost, latency, errors, volume, and eval performance, and remember your baselines between runs – so a persistent regression only shows up in a single report, not every day. AI Observability has [a built-in scout](/docs/self-driving/scout-examples.md) ready to turn on.

Create one from the [**Self-driving** tab](https://app.posthog.com/ai-observability/self-driving) in AI Observability or from your [inbox](https://app.posthog.com/inbox): pick a template like the daily digest, or describe what's worth your attention in plain English.

## What lands in your inbox

A [report](/docs/self-driving/reports.md) gives a summary and the evidence plus how actionable and important it is. If a fix is possible in code and the codebase is connected, one click opens a pull request. Reports are free; pull requests are [priced per PR](/docs/self-driving/pricing.md).

## Next step

See the full picture of the loop your LLM data is feeding.

[Self-driving](/docs/self-driving.md)

### Still have questions?

Ask PostHog AI

### Was this page useful?

HelpfulCould be better