What is a Scout? A technical deep dive

A scout is a small agent (don't overthink that word) that looks at, wrangles, and makes sense of any data available in PostHog. It's a core component of Self-driving in PostHog (now in open beta), an autonomous loop that monitors your product for patterns and drafts fixes.

A scout's job is to emit useful signals – things worth knowing – that you or your agents can act on. Sometimes that action happens automatically, like an agent opening a PR to fix what the scout found, sometimes it needs some human judgement before you and your agent can tackle it together.

A scout is, at its core, a skill – and that skill is the main interface everyone works through, human or agent. The business logic that used to be a sprawl of if/else branches and helper functions now starts life as a few hundred lines of English anyone can read, write, and improve (obligatory Karpathy tweet).

You can't get away with this for every data pipeline – plenty still needs determinism – but where you have the flexibility, the trade-offs are increasingly worth it.

This post walks through how scouts work in PostHog, with two real scouts we run on our own project – one that watches agent-driven MCP feedback, and one that watches what the internet says about us.

This is the deep dive. If you just want the short version, the Self-driving docs walk the whole self-improving loop – scouts, signals, reports, and the inbox – at a glance.

If you just want to see scouts in action, skip to Two example scouts, explained end to end – there's an ~8 minute video walkthrough of a real scout run there too.

The scout's role in PostHog's self-driving pipeline

Scouts don't live on their own, they feed our self-driving product pipeline:

1. A scout runs on a schedule, looks at some slice of your PostHog data, and decides whether anything is worth surfacing.
2. If it is, the scout emits a signal – a semi-structured finding with evidence, and a suggested action.
3. Signals get picked up by the self-driving pipeline, grouped, and turned into reports that land in your PostHog Inbox.
4. Reports get actioned – by you, or automatically by a PostHog Code task that picks up the report and ships a fix or improvement.

A report can be made up of many signals from many sources. Scouts are just one of those sources.

A single report – say "this MCP tool keeps blowing the token budget" – might be assembled from a scout finding, error tracking, a few raw events, and a health check, all grouped together because they're about the same underlying problem.

Scouts are one (agent-pilled) way PostHog Self-driving detects there's something to do at all. They help make your product self driving by automating routine papercuts (great for full automation), so you can focus more on strategic judgement work (firmly human still).

How scouts work

There are really only two parts to a scout: the skill that holds all the logic (mostly markdown), and a thin harness that runs it on a loop. Here's each in turn.

Your pipeline can be (mostly) markdown

In a traditional data pipeline, the logic that decides "is this interesting?" lives in code: thresholds, if/else branches, helper functions, a scheduler, a pile of glue. Changing what you care about means a code change, a review, a deploy. This is still needed, but for more subjective and flexible pipelines, the agent-pilled approach can offer many benefits.

With a scout, almost all of that logic lives in a skill – a markdown file (at a minimum) the agent is told to read and operate by. The skill is where you encode your business knowledge: what's normal, what's noise, what's worth waking a human for.

What used to be if response_rate < 0.3 and sample_size > 30: is now a sentence like "a flagship NPS survey needs ≥30 responses/week and a ≥10% score drop before it's worth acting on." That's trivially easy to iterate on and sense check with your agents, and you don't need to be an engineer to do it.

You can be as prescriptive as you like, but in many cases you can just tell the agent what you care about, give it a goal, and let it handle the actual logic. You can then fine tune your agent as it runs by adding dos and don'ts, and other parameters that guide its approach.

Everything else – the loop, the sandbox, the scheduler, the memory – is deliberately thin plumbing around that skill. The scout harness prompt, for example, is already too long at 200-ish lines – I need to chip away at that!

"A markdown file" is the lowest possible bar to entry – and for a lot of scouts it's genuinely all you need. But a scout is also a full Agent Skill, so it scales up the same way any skill does. The SKILL.md is the entry point, and a more complex scout can bundle reference files the agent loads on demand and runnable example scripts for the parts that genuinely want code.

So the spectrum runs from "a few sentences of English" up to "a skill with reference docs and executable scripts" – and the nice part is you start at the easy end and only reach for the heavier machinery when a particular scout actually needs it.

The really nice thing here is that it's easy to get a feel for this as you can just ask your agent how a scout is performing for your project using the exploring-signals-scouts skill we ship with the PostHog AI Plugin.

Or even easier just press one of the suggested questions here on the scouts page in PostHog Code. "Make a scout" will use the authoring-signals-scouts skill, scan your project and suggest some custom scouts for you.

The scout harness

We have a scout harness (yes, I used "harness" – cross it off your bingo card) that is really just a small layer running your scout skills in an agentic loop.

The main ingredients are:

The bulk of what actually makes a given scout is its skill – the markdown the agent is told to follow. There are a few small config knobs too (how often it runs, whether it's allowed to emit yet), exposed via the scout-config MCP tools.

The "special" tools that make a scout more than a one-shot loop

On top of the normal read-only MCP, a scout gets a handful of scout-only tools:

That's it. The scout reads existing inbox reports and prior runs through plain MCP tools; the only genuinely privileged things it can do are write to its scratchpad and emit signals.

Dogfood it yourself, then let PostHog run it

Because a scout is just a skill plus the MCP, you can run one manually in any agent surface where the PostHog MCP is available – PostHog Code, Claude Code, Codex, Cursor, whatever.

This is a key feature, not an accident: you can dogfood a scout on your own project, watch it work against your real data, and tweak the scout until it's good – grounded in actual usage and the pain points you hit on the way.

The thing that's a little different – and makes all the difference – is what happens when PostHog runs your scouts for you.

On the scheduled, sandboxed runs, the scout is allowed to maintain its scratchpad and emit signals into the pipeline. That permission is what turns a stateless one-shot loop into a long-running agent that learns, adapts, and evolves as your data does – while you steer it just by maintaining the one skill it runs.

Signal vs noise: What makes a scout good?

The hard part of a scout isn't finding things. It's not finding things. A scout that emits on every wobble is worse than no scout at all, because it trains you to ignore them.

So the interesting logic in a scout skill is mostly about restraint, such as:

• A sense of when to speak up. A scout's skill tells it to only emit when it's confident the finding is real and worth your attention – otherwise it writes a note to its scratchpad and moves on. Exactly how that bar is expressed is up to each scout, and it's a convention rather than a hard gate, so you can tune how trigger-happy a given scout is.
• Dedupe keys. Findings carry stable keys so the pipeline can group repeats and the scout can recognize "I already said this."
• Disqualifiers. Explicit lists of things that look like signal but aren't: a single user, a dev-environment burst, a survey at the natural end of its window, an internal test response.
• Severity (P0–P4) so downstream routing knows what's a five-alarm fire versus a nice-to-fix.
• ...anything else you decide!

Here's the contrast that makes it concrete. On our brand-mentions scout, a lone "PostHog is down" tweet – no detail, no corroboration, one account – got held below the bar.

The same week, a "PostHog is getting expensive" complaint that showed up from multiple independent voices got emitted as a pricing-perception theme. Same surface, opposite decisions, and the reasoning for both is written in plain English in the skill and the scratchpad.

The scratchpad: How a scout remembers

The scratchpad is a simple key/value store the scout reads at the start of a run and writes to as it goes. It's deliberately low-tech, but it's where the "learning" actually lives. Over time a scout grows a small, legible memory made of a few entry types:

Again you can steer your scout to take a different approach to memory here as works best for you.

This is the second half of the markdown thesis: not only did the if/else become English, the database row became a scratchpad note the agent reads and rewrites every run. You can read a scout's entire scratchpad and understand exactly what it knows and why it's been quiet.

Making a scout

A scout is 99% a skill, so making one is mostly writing (and iterating on) a skill. You don't do that by hand – you have your agent use the authoring-signals-scouts skill, which walks it through the anatomy: the emit contract, dedupe and memory conventions, disqualifiers, and the schedule config.

PostHog ships with around 20 canonical scouts out of the box – they live in products/signals/skills/. Here's a sample and high-level things they look for:

You can also write your own custom scouts for anything you care about – again, just point your agent at authoring-signals-scouts. Here are some internal ones we've built recently to dogfood the system:

And to see how your scouts are doing, there's (you guessed it) an exploring-signals-scouts skill – it surveys the fleet, reads recent runs, inspects the scratchpad, and assesses each scout's health.

Two example scouts, explained end to end

This is all a little hand-wavy so far, so let's trace two real scouts we run on our own project – one at the bottom of the self-driving product pyramid (an auto-fixable code paper-cut) and one nearer the top (judgment work a human owns).

Prefer to watch? Here's an ~8 minute walkthrough of a real run of the MCP feedback scout below – me clicking through an actual scout run end to end, from what it read to what it emitted.

1. The MCP feedback scout (the code one)

signals-scout-mcp-feedback (you can see it's a fairly simple skill) watches a single event: mcp feedback submitted.

When an AI agent (Claude Code, Cursor, etc.) hits friction using our MCP – a confusing schema, a tool that returns the wrong thing, a missing capability – it can file a structured report via the MCP's own agent-feedback tool.

That report becomes an event. The scout's job is to turn that low-volume, high-signal stream into actionable findings for the MCP team.

Here's one finding the whole way through that pipeline – a single agent's gripe becoming a PR. Each hop is a box: skim the summaries for the shape, or pop one open to see the actual artifact.

That's the whole loop this post is about: one piece of agent feedback, turned into a reproducible finding with a verbatim fix, turned into a root-caused report, and picked up by another agent – no human in the chain until review.

Those boxes trace one finding end to end. Two behaviors from this scout generalize beyond it, and both show the system learning:

2. The brand mentions scout (the non-code one)

signals-scout-brand-mentions (you can read it as a gist) watches what the internet says about PostHog. And the way it gets that data is the point.

Anything you can land in PostHog is scout-able. We pipe a social-listening tool (Octolens) into a Slack channel, and that channel is synced into the PostHog data warehouse as a regular table.

The scout just queries it with SQL like any other PostHog data. There's nothing special about "analytics events" here – warehouse sources, Slack relays, third-party feeds, all of it becomes fair game for a scout the moment it's in PostHog. This is a big part of why scouts are general-purpose rather than a product-analytics-only feature.

This scout sits higher on the self-driving pyramid than the MCP one: when it finds something, the action usually isn't a PR – it's comms, product strategy, or just situational awareness. A few real (anonymized) examples it surfaced:

Pop these open for the actual public roasts and the scout's clinical reason for holding them:

All real, all public, all genuinely useful to know in aggregate – and all held back until they actually matter.

And this is the scout whose scratchpad best shows the memory model evolving. Across runs it has built up:

• a pattern:brand_mentions:cursor entry it overwrites every run (the rolling "I've read up to here" mark),
• a stack of *-belowbar carries – "a user says PostHog's bot-filtering is letting bots through; n=1, low reach – promote if a second independent voice reports the same,"
• addressed:* entries recording every theme it's already emitted and the exact bar for re-emitting,
• and its own tags:brand_mentions:taxonomy vocabulary, with entries cross-linked by [[name]] references.

You can read that scratchpad top to bottom and reconstruct everything the scout knows, every judgment call it's made, and why it's been quiet on the things it's chosen to stay quiet on.

That legibility – being able to audit an agent's memory in plain text – is, to me, the whole pitch.

Ideas: what else to point a scout at

Those two are just two shapes. The thing to internalize is that anything you can land in PostHog is scout-able, and a scout's whole job is encoded in a skill you can write in an afternoon. Some directions to steal – most of these are scouts we already run on our own project:

The pattern across all of these: pick a surface you wish someone was watching, write down what "worth my attention" means for it in plain English, and let the scout hold that bar for you.

Where this is going

Scouts today are deliberately narrow: read your data, decide if something's worth knowing, emit a signal. That's the foundation, and keeping it small is the point. But there's a lot of obvious headroom, and a bunch of it is already on the roadmap:

The pattern here is to lean in and embrace being agent pilled, keep the core tiny and the skill in charge, and let scouts climb the self-driving pyramid – from noticing, to recommending, to doing.

Try it yourself

Scouts are still rolling out gradually and are invite-only for now while we shake out the rough edges – wider access is coming soon. If you want in early, email team-self-driving@posthog.com and we'll get you set up.

Once they're available to you, the home base is PostHog Code, our desktop coding agent – download it, sign in, and the PostHog MCP is wired up for you. From there you can navigate to the Agents section and explore the set of canonical scouts PostHog ships with as well as make your own custom scouts.

AGI Buzzword bingo

We hit a lot of AGI-era buzzwords on the way here. If you've been keeping score, here's your card – how many did we land?

(p.s. flip a card for its real definition – and the ✗'s are the few we never actually managed to get in)