Using sandboxed agents

Contents

Sandboxed agents are background AI agents that run in isolated cloud containers with access to PostHog data, GitHub repositories, and code execution. Use them when your feature needs an autonomous agent that reads PostHog data, writes code, and produces artifacts like PRs or reports.

For simpler LLM calls (summarization, translation, classification), skip this page and use the LLM gateway (get_llm_client(product=..., team_id=...)) directly — it's simpler and doesn't need a sandbox.

When to use what

ExampleSolution
Signals team building an enrichment pipeline that generates reports from PostHog analytics dataSandboxed agent (this page)
Conversations team building a support agent that queries PostHog and customer documentationSandboxed agent (this page)
AI observability summarizing a funnel, generating a natural-language insight titleLLM gateway via get_llm_client(product=..., team_id=...)
Not sureAsk in #team-posthog-ai

Rule of thumb: if the LLM needs to do things (query data, read files, create branches, open PRs), use a sandboxed agent. If it just needs to answer a question given some context you already have, use the LLM gateway.

How it works

A sandboxed agent runs inside an isolated cloud container (Modal in production, Docker locally). The system provisions the sandbox, clones a GitHub repo, starts an agent server, and waits for the agent to finish.

text
Your product code
│ Task.create_and_run(...)
Temporal workflow (process-task)
├── 1. Create scoped OAuth token
├── 2. Provision sandbox (Modal / Docker)
├── 3. Clone repository
├── 4. Start agent server
├── 5. Wait for completion (heartbeat-extended timeout)
└── 6. Cleanup sandbox

The agent inside the sandbox gets:

  • A scoped OAuth access token for the PostHog API (6-hour TTL)
  • A GitHub installation token for repo operations
  • Access to the PostHog MCP server for querying data
  • Code execution capabilities within the sandbox

Creating a sandboxed agent

Use Task.create_and_run() to launch a sandboxed agent from your product code:

Python
from products.tasks.backend.models import Task
task = Task.create_and_run(
team=team,
title="Generate weekly signal report",
description="Analyze error trends and generate a summary report with recommendations.",
origin_product=Task.OriginProduct.ERROR_TRACKING, # or your product's origin
user_id=user.id,
posthog_mcp_scopes="read_only", # or "full" if the agent needs write access
)

Parameters

ParameterRequiredDescription
teamYesThe team this task belongs to
titleYesHuman-readable task title
descriptionYesDetailed description of what the agent should do
origin_productYesWhich product created this task (see Task.OriginProduct choices)
user_idYesUser ID — used for feature flag validation and creating the scoped API key
repositoryYesGitHub repo in org/repo format (e.g., posthog/posthog-js)
posthog_mcp_scopesNoScope preset or explicit scope list (default: "full")
create_prNoWhether the agent should create a PR (default: True)
modeNoExecution mode (default: "background")
slack_thread_contextNoSlack thread context for agents triggered from Slack
start_workflowNoWhether to start the Temporal workflow immediately (default: True)
sandbox_environment_idNoID of a SandboxEnvironment to apply network restrictions (see below)

Adding a new origin product

If your product doesn't have an OriginProduct entry yet, add one to Task.OriginProduct in products/tasks/backend/models.py:

Python
class OriginProduct(models.TextChoices):
ERROR_TRACKING = "error_tracking", "Error Tracking"
# ...
YOUR_PRODUCT = "your_product", "Your Product"

Then create and run a Django migration.

Fine-grained access tokens

Every sandboxed agent gets a scoped OAuth access token that controls what PostHog resources it can access. Tokens expire after 6 hours and are scoped to a single team.

Scope presets

Use the posthog_mcp_scopes parameter to control access:

PresetWhat it grantsWhen to use
"read_only"Read access to actions, cohorts, dashboards, experiments, feature flags, insights, queries, surveys, etc.Agent only needs to read data for analysis or reporting
"full" (default)Read + write access to all MCP-exposed resourcesAgent needs to create or modify PostHog resources (e.g., create feature flags, update experiments)

Custom scopes

For more granular control, pass an explicit list of scopes instead of a preset:

Python
Task.create_and_run(
# ...
posthog_mcp_scopes=["query:read", "feature_flag:read", "experiment:read"],
)

Available read scopes: action:read, cohort:read, dashboard:read, error_tracking:read, event_definition:read, experiment:read, feature_flag:read, insight:read, project:read, query:read, survey:read, and others.

Available write scopes: action:write, cohort:write, dashboard:write, experiment:write, feature_flag:write, insight:write, survey:write, and others.

Internal scopes (task:write, llm_gateway:read) are always added automatically.

See posthog/temporal/oauth.py for the full list.

Principle of least privilege: default to "read_only" unless your agent genuinely needs to create or modify resources. This limits blast radius if the agent misbehaves.

PostHog MCP server

The sandbox comes with access to the PostHog MCP server, which exposes PostHog resources as tools the agent can call — listing feature flags, running HogQL queries, searching session recordings, etc.

The MCP server is ready to use today. For details on available tools, see Implementing MCP tools.

Skills

Skills are job-to-be-done templates that teach agents how to compose MCP tools into workflows. They provide domain knowledge, query patterns, and step-by-step guidance.

Skills are pre-installed in the sandbox base image and available to all sandboxed agents. They're copied to three discovery locations during image build:

  • /scripts/plugins/posthog/skills/ – plugin discovery
  • ~/.agents/skills/ – Codex agent discovery
  • ~/.claude/skills/ – Claude Code CLI discovery

For details on writing skills, see Writing skills.

Multi-turn sessions

MultiTurnSession provides a structured API for building custom multi-turn research agents. Use it when your agent needs multiple conversation turns with schema-validated responses – for example, a discovery pass followed by per-item research, then assessment and summarization.

Mental model

  1. StartMultiTurnSession.start(prompt, context, model=Shape) launches a sandbox, sends the first prompt, and returns a validated Pydantic model.
  2. Follow upsession.send_followup(prompt, Shape) sends additional prompts within the same sandbox session. Each response is validated against the provided schema. The session retries once on empty responses.
  3. Endsession.end() signals the sandbox workflow to shut down.

Example

Python
from products.tasks.backend.logic.services.custom_prompt_multi_turn_runner import MultiTurnSession
from products.tasks.backend.logic.services.custom_prompt_internals import CustomPromptSandboxContext
# 1. Start: discovery turn
session, candidates = await MultiTurnSession.start(
prompt="Find up to 10 items to investigate.",
context=context, # CustomPromptSandboxContext
model=DiscoveryResult,
branch="master",
step_name="my_discovery",
)
# 2. Follow up: research each item
for item in candidates.items:
finding = await session.send_followup(
f"Research {item.name} in detail.",
FindingResult,
label=f"research_{item.name}",
)
# 3. Follow up: summarize
summary = await session.send_followup(
"Summarize all findings.",
SummaryResult,
label="summary",
)
# 4. End the session
await session.end()

Reference implementation

See products/tasks/backend/logic/services/mts_example/ for a complete working example. It runs a multi-turn agent that discovers "cursed" identifiers in a repo, researches each one, and produces output in the shape Signals consumes:

text
discovery → research ×N → actionability → priority? → presentation

Run it locally (DEBUG only):

Terminal
DEBUG=1 python manage.py demo_mts_example --team-id <id> --user-id <id>

See the example README for details on adapting it to your own use case.

Code execution

Agents run inside an isolated sandbox with full code execution capabilities. They can:

  • Read, write, and execute files in the cloned repository
  • Install dependencies (npm, pip, etc.)
  • Run tests, linters, and build tools
  • Create git branches and pull requests (commits must be signed — see below)
  • Execute arbitrary shell commands within the container

Git commit signing

Direct git commit and git push commands are blocked in the sandbox to ensure all commits are properly signed by GitHub. A PATH shim (git-guard.sh at /opt/posthog/bin/git) intercepts these subcommands while passing all other git operations through to the real binary.

If an agent attempts to run git commit or git push, it will see:

text
git commit is disabled in PostHog Desktop: commits must be signed.
To commit: stage changes with 'git add', then call the git_signed_commit tool.
To force-push after a rebase/conflict fix: call the git_signed_rewrite tool.

Agents should stage changes with git add, then use the git_signed_commit tool to create signed commits. For force-pushing after a rebase or conflict resolution, use the git_signed_rewrite tool instead.

Debugging escape hatch: Set POSTHOG_ALLOW_UNSIGNED_GIT=1 in the sandbox environment to bypass this restriction. This is intended for debugging only and should not be used in production.

Sandbox isolation

Production (Modal)Local dev (Docker)
IsolationgVisor container or VM microVMStandard Docker container
NetworkConfigurable via SandboxEnvironmentHost network via host.docker.internal
Imageghcr.io/posthog/posthog-sandbox-baseLocal Dockerfile build
AuthModal connect tokenNo token needed

Runtime selection (gVisor vs Modal VM)

Production sandboxes run on one of two Modal runtimes, chosen per run in get_task_processing_context (_resolve_modal_vm_sandbox) and forked in provision_sandbox:

  • gVisor (SandboxTemplate.DEFAULT_BASE) — the historical default: a gVisor kernel-sandboxed container.
  • Modal VM (SandboxTemplate.VM_BASE) — a kernel microVM that also bakes in Docker-in-Docker, so the agent can run nested containers. Custom base images layer on this base, and it is what image-builder runs execute on.

Selection is driven by the tasks-modal-vm-sandbox flag's JSON payload, which carries two origin allowlists and an optional default image:

  • origin_products — origins allowed on the VM runtime when a custom image is resolved for the run (custom images cannot run under gVisor).
  • default_base_origin_products — origins that default to the bare VM base image even without a custom image. This is the knob for making the VM runtime the default for standard cloud runs; we widen it origin-by-origin (and, later, the flag's release condition) as the rollout expands.
  • default_custom_image — a Modal image name that VM runs fall back to when no custom image was picked. Because the flag's payload variants are org-targeted, this routes which default VM image an org gets: PostHog's own org points at the prebaked dev-stack image (below), everyone else keeps the plain VM base. A user- or environment-selected custom image always wins over this default, and provisioning falls back to the plain VM base if the named image is missing.

The prebaked dev-stack image

hogli start on a fresh VM pays for multi-gigabyte docker pulls and the full Django + persons + ClickHouse migration history — and dead-ends anyway, because the lean VM base lacks the dev toolchain flox provides on dev machines (brotli, phrocs, Go, Rust). For runs on the PostHog monorepo we bake all of that ahead of time: the bake-dev-stack-image Temporal workflow boots a plain VM-base sandbox, runs bake-posthog-dev-stack.sh inside it (install the dev toolchain, pre-pull the dev compose images, bring the stack up, run the Django and Rust-driven migrations, shut down cleanly), snapshots the filesystem, and publishes it under the fixed Modal image name posthog-dev-stack (see products/tasks/backend/logic/services/dev_stack_image.py). It is dispatched on two cadences, both gated per region on the tasks-dev-stack-image-bake flag via a region person property: a nightly full rebake that keeps the heavy state (migrations, docker pulls) close to master, and a two-minute sweep that rebakes as soon as the VM base image digest moves (e.g. an agent-server release), at most once per new digest. User-authored custom images retain their separate ten-minute, batched refresh fanout. python manage.py bake_dev_stack_image triggers a bake manually and bypasses the flag. Pointing an org's default_custom_image payload key at that name gives its VM runs warm docker state and already-migrated databases, so a task-time hogli start only applies the migrations that landed since the last bake. The pnpm store and Playwright's Chromium are prewarmed too: pnpm install --frozen-lockfile --prefer-offline is a fast linking pass and browser installs are no-ops. Build outputs (node_modules, Storybook dist, Vite/Turbo caches) are deliberately not baked — the bake's checkout is deleted before the snapshot — so frontend builds always run from the task's own source. The bake must run on the real VM runtime — dockerd cannot run inside Modal's gVisor image builder — which is why it is a sandbox filesystem snapshot rather than a spec-built image.

At task time the restored image is not self-starting: the sandbox runtime rewrites /etc/hosts at boot and dockerd does not autostart. Run the baked bootstrap-dev-stack helper first (restores the compose host aliases and starts dockerd — the bake manifest at /opt/posthog/dev-stack-bake.json names it under bootstrap), then from the checkout run uv sync, source .venv/bin/activate, hogli start -y -d, and hogli wait. Detached mode is required — the sandbox has no TTY, and running phrocs under a pseudo-TTY makes it balloon in memory until it is OOM-killed — and the detached start returns while the stack is still booting, so hogli wait is what blocks until every process reports ready.

Provisioning also fires that helper detached as soon as the sandbox is up (best-effort, only on runs that booted the PostHog-published posthog-dev-stack image itself — never a user-authored custom image, and never a filesystem-snapshot restore, whose filesystem a prior run could have altered; directory-snapshot resumes only mount the workspace, so they keep the warmup), so the dockerd warmup overlaps the repo clone and the environment is usually ready by the agent's first command. Running bootstrap-dev-stack again is still the right first step — it is the synchronization point, blocking until the warmup completes.

Restricted runs can use the VM runtime only when tasks-modal-network-allowlist is also enabled. The network flag interlock runs before state overrides, image-builder routing, custom-image routing, and the VM rollout flag. A trusted use_modal_vm_sandbox state value cannot bypass it. Modal is the authoritative network enforcement layer whenever that flag is enabled, including on VMs. AgentSH also applies the compiled policy to the agent-server process tree as defense in depth. The provider policy applies outside the sandbox, so it covers traffic from the VM and its Docker containers without relying on AgentSH process tracing. Modal applies domain restrictions using the requested hostname or TLS SNI. Raw IP connections without an allowed SNI fail, while a connection to an IP with an allowed SNI can pass the boundary. This is an SNI allowlist, not DNS-to-destination-IP binding. AgentSH repeats the domain policy for the processes it traces as a second layer. It is not authoritative for VM egress: traffic that bypasses its proxy or process tree is outside that layer, and AgentSH does not add strict hostname-to-destination binding. Use an externally enforced egress proxy with a provider CIDR allowlist if a workload needs that binding. Test both host and container traffic because their network paths differ. The use_modal_vm_sandbox run-state key force-selects the VM runtime for trusted server-created runs (image builders) and is never accepted from client input.

Network access

Network access is configured per-team via SandboxEnvironment:

  • Trusted — only allows access to a default set of trusted domains (GitHub, npm, PyPI, etc.)
  • Full — unrestricted network access
  • Custom — explicit allowlist of domains, optionally including the trusted defaults

Allowed-domain values contain a domain name only, such as example.com or *.example.com. Schemes, paths, ports, IP addresses, rooted names, local host aliases, and wildcards in other positions are rejected when an environment is created or updated. Values are normalized to lowercase IDNA names, duplicates are removed, and each environment can contain up to 100 allowed domains.

None is the internal representation for unrestricted access. A restricted empty list still includes the infrastructure domains required to run the sandbox. Modal and agentsh consume provider-specific forms of one compiled effective policy, including the same infrastructure domain coverage.

To apply network restrictions from your product code, create a SandboxEnvironment and pass its ID to Task.create_and_run:

Python
from products.tasks.backend.models import SandboxEnvironment, Task
# 1. Create an environment (once, or look up an existing one)
env = SandboxEnvironment.objects.create(
team=team,
created_by=user,
name="Restricted agent env",
network_access_level="custom", # "full" | "trusted" | "custom"
allowed_domains=["github.com", "api.example.com"],
include_default_domains=True, # merge GitHub, npm, PyPI defaults
)
# 2. Pass its ID when creating the task
task = Task.create_and_run(
team=team,
title="My restricted task",
description="...",
origin_product=Task.OriginProduct.YOUR_PRODUCT,
user_id=user.id,
repository="org/repo",
sandbox_environment_id=str(env.id),
)

The temporal workflow resolves and compiles allowed domains at execution time, so environment updates take effect on the next run. The compiled policy and its fingerprint stay fixed across activity retries. Modal enforces the network boundary on every restricted run when tasks-modal-network-allowlist is enabled. During the rollout, restricted runs without that flag stay on gVisor and use agentsh; they cannot route to a VM without the provider policy.

Environments can also be managed via the REST API (SandboxEnvironmentViewSet) or the PostHog Desktop settings UI.

Custom base images

Teams can bake their own tools and dependencies into a custom base image (SandboxCustomImage) and select it as a cloud environment's base via SandboxEnvironment.custom_image. Custom images always layer on top of the published VM sandbox base — agent tooling, git guard, and the VM runtime are always present — and the whole mechanism is gated on the Modal VM runtime being available: the sandbox_custom_images API returns 403 (and the PostHog Desktop UI hides the feature) unless the tasks-modal-vm-sandbox flag is enabled for the org with user_created in its origin_products payload allowlist, since custom-image sandboxes cannot run under gVisor.

The flow, driven from the PostHog Desktop Environments → Cloud tab:

  1. Creating an image spawns an interactive image-builder agent task (custom_image_builder_id in the run state, VM runtime forced) that iterates inside the real VM base and maintains a declarative spec (SandboxImageSpec: apt_packages, run_commands, env) at /tmp/workspace/image-spec.yaml.
  2. "Save & build" reads the spec from the builder sandbox (or accepts it inline via POST /api/projects/:id/sandbox_custom_images/:id/build/), then the build-sandbox-image Temporal workflow runs an LLM security scan of the spec, builds it layered on the VM base, and publishes it as a Modal named image (Image.publish() / Image.from_name()).
  3. Runs using an environment with a ready custom image provision their sandbox from the published image (SandboxConfig.custom_image_name), falling back to the standard base if the image can't be loaded. Repo-setup snapshots are skipped for custom-image runs; resume snapshots still apply.

Local development

To set up sandboxed agents for local development:

  1. Create a personal dev GitHub App (see the Cloud runs setup guide for details)
  2. Run python manage.py setup_background_agents
  3. Run hogli start

The setup command is idempotent and handles:

  • Writing required env vars (OIDC_RSA_PRIVATE_KEY, SANDBOX_JWT_PRIVATE_KEY, DEBUG, SANDBOX_PROVIDER, SANDBOX_MCP_URL) to your .env
  • Creating the Array OAuth application
  • Enabling the tasks feature flag for all teams
  • Building the agent skills bundle

For advanced setup options (Modal sandboxes, local agent packages, MCP), see the Cloud runs setup guide.

Tip: Set SANDBOX_REPO_MOUNT_MAP to bind-mount local repositories into the Docker container and skip cloning from GitHub. Format: SANDBOX_REPO_MOUNT_MAP=org/repo:/local/path (e.g., SANDBOX_REPO_MOUNT_MAP=PostHog/posthog:~/Developer/posthog). This can significantly reduce sandbox startup time for large repos.

Questions?

If you're unsure whether a sandboxed agent is the right fit for your use case, ask in #team-posthog-ai on Slack.

Was this page useful?