Connect a deploy to an error burst

Twig ships several times a day. The PR follow-up scout ties a new issue after a deploy to the pull request that likely caused it, by matching its stack trace against the files each pull request changed.

What the scout looks at

Twig's release 1.4.2 went out at 09:10 with five pull requests. Four minutes later, a new TypeError started firing on Trip details Fig. 1. Its stack frames are in a file one of those pull requests changed, so the scout reports it against that pull request. A new issue with no frames in changed files goes to the spikes and stuck loops scout instead.

1.4.2 deployed

Sessions

unchanged

Trip details exceptions

starts in 1.4.2

Fig. 1 – What the scout saw: Release 1.4.2 went out at 09:10. Trip details exceptions climbed after it while sessions held steady.

An older issue counts too if it reopens or fires at a clearly higher rate after the deploy.

The report in your inbox

The report names the pull request, its author, merge time, and deploy time, compares the error before and after, and recommends fixing the code or rolling back Fig. 2.

P2New Trip details TypeError since the 1.4.2 deploy, thrown from a file twig.com#4877 changed A new TypeError on Trip details started with the 1.4.2 deploy. Its top frame is in a file twig.com#4877 changedScout · PR follow-upReadyActionable2h ago
ArchiveReview
Fig. 2 – This report card lands in Twig's inbox. It shows the scout's claim and triage, and the evidence is inside the report.Hover over the figure to learn about each element.

The scout itself

PostHog maintains this scout. Its skill file is on GitHub. It waits for each merged pull request to deploy. Then it matches the stack frames of issues that started in the deploy window against the files the pull request changed. It reads deploy times from GitHub deployments, GitHub releases, or deploy annotations. Without one, it estimates 24 hours after the merge for server code, and 72 hours or more for web and mobile. It runs on the schedule the engineers set in its Settings, checks about eight pull requests each run, and stays quiet unless something needs the team's attention.

Add this to your scout troop

The engineers turn it on for each project. It uses:

  • Required: GitHub connected to PostHog, so the scout can see merged pull requests and the files they changed. See GitHub.
  • Recommended: a deploy signal, like GitHub deployments synced through the GitHub source. Without one, the scout estimates deploy times.
  • Recommended: source maps for bundled code, so stack frames name the source files. See Stack traces.
Open your scoutsFind PR follow-up in the list and check it's on.