Tech stack
Contents
Note: This page refers to our main product repository, not our website.
Frontend
- Web framework/library: React
- State management: Redux + Kea
- Layout/components: Ant Design
Backend
- Framework: Django
- High scale services: Rust
- Databases: PostgreSQL and ClickHouse
- Task queue/event streaming: Redis and Apache Kafka
- Task Worker: Celery, Temporal and Dagster
Testing
- Frontend E2E tests: Cypress
- Backend tests: Pytest and Django's built-in test suite
Additional tools
- CI/CD: GitHub Actions
- Containerization: Docker and Docker Compose
- Linter (frontend): ESLint
- Formatter (backend): Black
Workflow orchestration
We used to use Celery as our task worker, but it has consistently proven to be unreliable at our scale. We have legacy jobs running on Celery, but new jobs should NOT use it, and teams should consider moving away from it for critical applications.
We use both Temporal and Dagster for different types of workflow orchestration, each chosen for their specific strengths.
When to use each tool
We tend to use Dagster for internal jobs, whereas Temporal is used for user-facing jobs. This is not a one-size-fits-all solution, and you can follow a different approach.
You can look at the problem from the requirements for your jobs:
- Is it mission-critical with complex failure scenarios? → Temporal
- Do you need exactly-once guarantees? → Temporal
- Do you need complex retry policies? → Temporal
- Is it a long-running stateful workflow? → Temporal
- Is it primarily about data transformation and dependencies? → Dagster
- Do you need rich data lineage and testing? → Dagster
Where do we use Temporal and Dagster?
These are examples of where we use Temporal and Dagster at PostHog. Hopefully, these can serve as anecdotal examples to help you pick between Temporal and Dagster for your application. This list is not exhaustive.
Temporal: Batch exports, data warehouse source syncing, AI platform task generation
Dagster: Exchange rate tracking, one-off production management commands (better monitoring than Django's management commands), web analytics data pre-processing