Linking Firecrawl as a source

Alpha release

This source is currently in alpha. The interface and available tables may change.

Firecrawl is a web scraping and crawling API. This connector syncs your Firecrawl account's operational data – job activity, credit and token usage, active crawls, and change-detection monitors – into the PostHog Data warehouse, so you can track your scraping usage and spend alongside the rest of your data.

Prerequisites

A Firecrawl account.
A Firecrawl API key, which you can create in your Firecrawl dashboard. A single key grants access to every table this connector syncs.

Adding a data source

In PostHog, go to the Sources tab of the data pipeline section.
Click + New source and click Link next to this source.
Enter your credentials (see Configuration below) and click Next.
Select the tables you want to sync, choose a sync method and frequency, then click Import.

Once the syncs are complete, you can start querying this data in PostHog.

When linking Firecrawl, you'll need:

API key – create one under API Keys in your Firecrawl dashboard. It starts with fc-.

Sync modes

Each table can be synced in one of several modes, depending on what the source supports:

Webhook (when available) – the source pushes changes to PostHog in real time. Fastest freshness, lowest ongoing cost, and the only mode that reliably captures updates and deletes.
Incremental – only new or updated rows are synced on each run, using a cursor field (such as an updated_at timestamp). Cheaper than a full refresh, but deletes aren't captured.
Append only – new rows are appended using a cursor field; existing rows are never updated. Ideal for immutable, append-only tables like event logs.
Full refresh – the whole table is reloaded on every sync. Use it when a table has no reliable cursor or when you need deletions reflected.

See sync methods for a full explanation of how each mode works and how to choose between them.

All Firecrawl tables are full refresh only. Firecrawl's account endpoints don't expose a server-side "updated since" filter, so each sync replaces the table with the current data from the API.

Note: The team_activity table is a rolling log that Firecrawl only retains for the last 24 hours. Older activity can't be backfilled, so schedule frequent syncs if you want to accumulate a longer history of jobs in the warehouse.

Configuration

Option	Type	Required
`API key`	password	Yes

Supported tables

Table	Description	Sync method	Incremental field	Primary key
`team_activity`	Job activity log. Firecrawl only retains the last 24 hours, so sync frequently to accumulate history - older activity cannot be backfilled.	Full refresh	—	id
`credit_usage_historical`	Credits used per billing period, month by month.	Full refresh	—	startDate
`token_usage_historical`	Tokens used per billing period, month by month.	Full refresh	—	startDate
`active_crawls`	Crawls that are currently in progress for your team.	Full refresh	—	id
`monitors`	Change-detection monitors that periodically re-scrape targets and report what changed.	Full refresh	—	id
`monitor_checks`	Change-detection runs, one row per check across every monitor. Off by default: it fans out one request per monitor.	Full refresh	—	id

The monitor_checks table is disabled by default because it fans out one request per monitor. Enable it in the table picker if you use Firecrawl monitors and want their individual check runs.

Troubleshooting

If the connection fails to validate, confirm your API key is active in your Firecrawl dashboard and that you pasted the full fc-... value.
If team_activity is empty, the endpoint only returns the last 24 hours of jobs. If your account hasn't run any jobs in that window, the table will be empty until it does.
If you hit rate limits, the connector automatically backs off and retries. Firecrawl enforces plan-based rate and concurrency limits.

If your sync is failing or data looks wrong, see the Data warehouse troubleshooting guide. If that doesn't help, contact support – we're happy to help.