Linking LangSmith as a source
This source is currently in alpha. The interface and available tables may change.
Sync your LangSmith LLM observability data – traces and runs, tracing projects, datasets, examples, feedback, and annotation queues – into the PostHog data warehouse, so you can join LLM behavior and cost with your product analytics.
Prerequisites
You need a LangSmith account with access to the workspace you want to sync. Any member can create a personal access token; workspace-scoped service keys need workspace admin access.
Adding a data source
- In PostHog, go to the Sources tab of the data pipeline section.
- Click + New source and click Link next to this source.
- Enter your credentials (see Configuration below) and click Next.
- Select the tables you want to sync, choose a sync method and frequency, then click Import.
Once the syncs are complete, you can start querying this data in PostHog.
You need a LangSmith API key. Create one in your LangSmith settings under API Keys.
Leave the Host field blank if your account is on the US cloud (api.smith.langchain.com). Set it to https://eu.api.smith.langchain.com for EU-region accounts, or to your own host for self-hosted LangSmith deployments.
Sync modes
Each table can be synced in one of several modes, depending on what the source supports:
- Webhook (when available) – the source pushes changes to PostHog in real time. Fastest freshness, lowest ongoing cost, and the only mode that reliably captures updates and deletes.
- Incremental – only new or updated rows are synced on each run, using a cursor field (such as an
updated_attimestamp). Cheaper than a full refresh, but deletes aren't captured. - Append only – new rows are appended using a cursor field; existing rows are never updated. Ideal for immutable, append-only tables like event logs.
- Full refresh – the whole table is reloaded on every sync. Use it when a table has no reliable cursor or when you need deletions reflected.
See sync methods for a full explanation of how each mode works and how to choose between them.
The runs table can be large. Use incremental sync for it: runs are filtered server-side by start_time, and the first incremental sync only pulls the last 365 days. Each incremental sync also re-reads a short trailing window so runs that finish (or receive feedback) after they were first synced get updated.
LangSmith rate-limits its run query API, so very large workspaces may see the initial runs backfill take a while.
Configuration
| Option | Type | Required |
|---|---|---|
API key | password | Yes |
Host | text | No |
Supported tables
| Table | Description | Sync method | Incremental field | Primary key |
|---|---|---|---|---|
runs | Traces and their nested spans (LLM calls, chains, tools) across every tracing project. The first incremental sync only pulls the last 365 days | Incremental, Full refresh | start_time | id |
projects | Tracing projects (called sessions in the LangSmith API) | Full refresh | — | id |
datasets | Datasets used for testing and evaluating LLM applications: collections of example inputs and expected outputs. | Full refresh | — | id |
examples | Examples (input/output pairs) belonging to datasets, used as test cases and evaluation ground truth. | Full refresh | — | id |
feedback | Human and programmatic feedback scores attached to runs, e.g. thumbs up/down, correctness scores, or evaluator results. | Incremental, Full refresh | created_at | id |
annotation_queues | Annotation queues used to collect human feedback on runs, with reviewer and reservation settings. | Full refresh | — | id |
Troubleshooting
If the source fails to connect, check that the API key is valid and that the Host field matches your LangSmith region – an EU-region key against the default US host returns a 401 error.
If your sync is failing or data looks wrong, see the Data warehouse troubleshooting guide. If that doesn't help, contact support – we're happy to help.