Linking LlamaCloud as a source
This source is currently in alpha. The interface and available tables may change.
The LlamaCloud connector syncs your document-processing data from LlamaCloud (LlamaIndex's hosted platform) – parse, extract, classify, split, and spreadsheet jobs, batches, pipelines, projects, files, and usage metrics with credit consumption – into PostHog, so you can analyze your document pipelines and AI costs alongside your product data.
Prerequisites
You need a LlamaCloud account and an API key. Create one in LlamaCloud under Settings > API Keys. API keys are scoped to a project and to the region they were created in (North America or Europe).
Adding a data source
- In PostHog, go to the Sources tab of the data pipeline section.
- Click + New source and click Link next to this source.
- Enter your credentials (see Configuration below) and click Next.
- Select the tables you want to sync, choose a sync method and frequency, then click Import.
Once the syncs are complete, you can start querying this data in PostHog.
When linking LlamaCloud, you'll need:
- API key – the key you created in LlamaCloud (starts with
llx-). - Region – the region your API key was created in. A key only works against its own region's API, so a key from the wrong region fails validation.
Sync modes
Each table can be synced in one of several modes, depending on what the source supports:
- Webhook (when available) – the source pushes changes to PostHog in real time. Fastest freshness, lowest ongoing cost, and the only mode that reliably captures updates and deletes.
- Incremental – only new or updated rows are synced on each run, using a cursor field (such as an
updated_attimestamp). Cheaper than a full refresh, but deletes aren't captured. - Append only – new rows are appended using a cursor field; existing rows are never updated. Ideal for immutable, append-only tables like event logs.
- Full refresh – the whole table is reloaded on every sync. Use it when a table has no reliable cursor or when you need deletions reflected.
See sync methods for a full explanation of how each mode works and how to choose between them.
The job tables (parse_jobs, extract_jobs, classify_jobs, batches, split_jobs, sheets_jobs) support incremental sync on created_at. Each incremental sync also re-reads a trailing 24-hour window, so status changes to recently created jobs are picked up. Status changes to jobs created earlier than that are only reflected on a full refresh.
The usage_metrics table supports incremental sync on day, and re-reads the boundary day each sync so its still-accumulating totals stay up to date.
The projects, pipelines, and files tables don't expose a timestamp filter in the LlamaCloud API, so they sync with a full refresh.
Configuration
| Option | Type | Required |
|---|---|---|
API key | password | Yes |
Region | select | Yes |
Supported tables
| Table | Description | Sync method | Incremental field | Primary key |
|---|---|---|---|---|
parse_jobs | Incremental syncs filter on created_at, re-reading a trailing 24h window so recent status transitions are picked up; status changes to jobs created earlier than that are only reflected on a full refresh | Incremental, Full refresh | created_at | — |
extract_jobs | Incremental syncs filter on created_at, re-reading a trailing 24h window so recent status transitions are picked up; status changes to jobs created earlier than that are only reflected on a full refresh | Incremental, Full refresh | created_at | — |
classify_jobs | Incremental syncs filter on created_at, re-reading a trailing 24h window so recent status transitions are picked up; status changes to jobs created earlier than that are only reflected on a full refresh | Incremental, Full refresh | created_at | — |
batches | Incremental syncs filter on created_at, re-reading a trailing 24h window so recent status transitions are picked up; status changes to jobs created earlier than that are only reflected on a full refresh | Incremental, Full refresh | created_at | — |
split_jobs | Incremental syncs filter on created_at, re-reading a trailing 24h window so recent status transitions are picked up; status changes to jobs created earlier than that are only reflected on a full refresh | Incremental, Full refresh | created_at | — |
sheets_jobs | Incremental syncs filter on created_at, re-reading a trailing 24h window so recent status transitions are picked up; status changes to jobs created earlier than that are only reflected on a full refresh | Incremental, Full refresh | created_at | — |
projects | LlamaCloud projects visible to the API key. | Full refresh | — | — |
pipelines | Managed index pipelines that ingest, chunk, embed, and index documents for retrieval. | Full refresh | — | — |
files | Files uploaded to LlamaCloud for parsing, extraction, or indexing. | Full refresh | — | — |
usage_metrics | Per-day usage and credit consumption aggregates | Incremental, Full refresh | day | — |
Troubleshooting
If validation fails with an invalid API key error, check the Region setting first: LlamaCloud keys are region-specific, so a key created in the EU region is rejected by the North America API and vice versa.
If your sync is failing or data looks wrong, see the Data warehouse troubleshooting guide. If that doesn't help, contact support – we're happy to help.