In-depth: PostHog vs GrowthBook
Contents
PostHog and GrowthBook both provide open source, self-serve feature flags and experimentation, but they're different in two important ways:
GrowthBook is a warehouse-native feature flag and experiments platform. It focuses on integrating with the product and data tools you already use.
PostHog is an all-in-one product and data platform. Beyond feature flags and experiments, it includes product analytics, session replay, surveys, CDP, and more.
In this post, we'll cover these differences in more detail, and answer frequently asked questions about both tools.
How is PostHog different?
1. PostHog is an all-in-one platform
PostHog brings together all the tools engineers need for testing, releasing, and measuring the success of new features. Feature flags and A/B testing are only part of a suite of tools PostHog offers.
PostHog combines usage, performance, and behavioral data with flags and experiments. PostHog’s data warehouse also enables you to pull in data from external sources.
Having all these product and data tools together enables you to do better analysis of shipped features and make better decisions about what you are building next.
2. PostHog is built for startups and engineers
PostHog is built for high-growth startups. This means it is simple for founders and engineers to implement themselves. There are many SDKs, tutorials, and docs to help you get started quickly with any type of app.
As startups scale, PostHog also provides the more advanced tools they need to succeed. These include advanced product analytics, SQL querying, CDPs, and data warehousing.
GrowthBook, on the other hand, focuses only on later-stage, larger companies than PostHog. Many of their features, like their analytical A/B testing suite, are great for data teams which also come at a later stage.
3. PostHog is easier to set up
GrowthBook requires more setup than PostHog as it relies heavily on external tools and writing SQL:
- To get data into GrowthBook, you must integrate a data source.
- To track data related to flags and experiments, you connect to an analytics tool.
- To configure data sources and track events, you write SQL configurations in GrowthBook.
- To set up goals and analysis, you write more SQL.
PostHog needs none of this. You use the same app and SDKs for managing and evaluating feature flags as tracking usage. There is no extra configuration needed. Creating flags, experiments, and insights doesn’t require writing SQL. This makes PostHog faster to get started and easier to use once set up.
Comparing PostHog and GrowthBook
Platform
Both PostHog and GrowthBook have the infrastructure to use flags and experiments effectively with your current app. PostHog has a wider range of product and data tools built in, while GrowthBook relies on third-party integrations
PostHog | GrowthBook | |
Open source Code publicly accessible | ✔ | ✔ |
Self-service No need to talk to sales | ✔ | ✔ |
SDKs Including JavaScript, Android, iOS, and Python | 14 | 11 |
Native data sources Compute metrics and results without integrations | ✔ | ✖ |
Proxies Reverse proxy to avoid blockers | ✔ | ✔ |
Self-hostable Host on your own infrastructure | ✔ | ✔ |
API Edit and evaluate flags with API | ✔ | Beta |
Local evaluation (aka streaming) Faster flags by not having to rely on the server for evaluation | ✔ | ✔ |
SQL Query flag and product data directly via SQL | ✔ | ✔ |
Session replays Real user playback of flag session | ✔ | ✖ |
Web analytics Simple dashboard of aggregate traffic, sources, campaigns, and session metrics | ✔ | ✖ |
Feature flags
Both PostHog and GrowthBook offer all the functionality you expect from feature flags.
PostHog | GrowthBook | |
Boolean flags Simple flags returning true or flag | ✔ | ✔ |
Percentage rollouts Target percentages of a group | ✔ | ✔ |
Custom targeting Target users based on user properties, custom contexts | ✔ | ✔ |
JSON payloads Flags return JSON | ✔ | ✔ |
Environments Manage flags for dev, staging, prod | Partial | ✔ |
Scheduling Schedule flags to turn on or off | ✔ | ✔ |
Early access management Manage betas, test features | ✔ | ✖ |
Bootstrapping Flags available on frontend application load | ✔ | ✖ |
Targeting: GrowthBook's custom targeting using attributes must be set every session and defined in-app before use. It's also only available on the paid Pro plan. PostHog automatically sets its equivalent (properties) on users. You don’t need to pre-define them and unlimited custom values are free.
Bootstrapping: PostHog’s JavaScript web SDK enables you to pass flags directly from the backend before the app loads. This ensures they're available immediately and prevents flickering. GrowthBook instead recommends moving the A/B test or flag logic earlier in the page load (server-side) to prevent this.
Experimentation
Experimentation is where PostHog and GrowthBook’s functionality diverges. Both enable you to create A/B/n tests with custom goals and calculate statistical significance, but:
GrowthBook’s A/B testing provides more potential statistical depth, such as Bayesian and Frequentist engines, value capping, regression adjustment, and testing corrections. These do require statistics expertise to use reliably, though.
PostHog provides a simpler interface and no need to write SQL or connect to external services – a better experience for users without a background in data science.
PostHog | GrowthBook | |
Custom goals Customize metrics that test tracks | ✔ | ✔ |
Secondary metrics Monitor impact on unrelated metrics | ✔ | ✔ |
Native goal tracking Track impact on goals without external sources | ✔ | ✖ |
Statistical significance calculation Calculate if changes make a statistically significant impact | ✔ | ✔ |
Split testing Split participants into groups | ✔ | ✔ |
Multivariate (A/B/n) testing Test multiple variants of a change | ✔ | ✔ |
Recommended run time Calculate the recommended run time for your experiments | ✔ | ✖ |
Namespaces Avoid conflicts by assigning all users to a namespace | ✖ | ✔ |
Visual editor Create and edit experiments in your app | Beta | ✔ |
Statistics engine How the success of an experiment is calculated | Bayesian | Bayesian, Frequentist |
- Recommended run time: PostHog automatically calculates a recommended run time based on past data and minimally acceptable improvements. This helps you avoid the peeking problem and end your experiment at the right time.
Visual editor: GrowthBook’s visual editor is similar to PostHog’s toolbar. It enables you to select an element to modify using an A/B test. The editor is limited to server-side rendered apps.
Frequentist engine: GrowthBook has an additional statistical analysis option. It enables you to compare the control with the experiment using p-values and confidence intervals.
Reporting and analytics
GrowthBook offers no reporting or analytics tools other than the ability to query experiment reports with SQL. As a result, GrowthBook forces users to rely on external event tracking and visualization tools.
PostHog provides all the visualizations and product tools for evaluating the success of your features and app. These include trends, funnels, retention, SQL querying, and session replays integrated with your flags and A/B testing data. This enables you to do deeper analysis of their impact and combine them with other product and usage data.
PostHog | GrowthBook | |
Analytics Capture usage and calls related to flags | ✔ | ✖ |
Experiment reports Show the results of an experiment | ✔ | ✔ |
Trends Graphs of flag usage | ✔ | ✖ |
Funnel visualization Analyze impact of flags on funnels | ✔ | ✖ |
Retention User retention related to flags | ✔ | ✖ |
Breakdowns Breakdown reports by properties | ✔ | ✔ |
Dashboards Multiple customizable insights for a flag | ✔ | ✖ |
Non-flag data Combine flag data with other product data | ✔ | ✖ |
Pricing
PostHog | GrowthBook | |
Pricing | Per request | Seat-based |
Free plan | ✔ | ✔ |
Free experiments | ✔ | ✔ |
Free collaboration | ✔ | 3 seats free, then $20/seat/month |
Fully transparent | ✔ | ✖ |
PostHog’s feature flag pricing is pay-per-request (and A/B tests use feature flags). There is a generous free tier of 1M requests per month with all features, add-ons, and integrations available.
Like PostHog, GrowthBook is free to self-host. GrowthBook Cloud is a seat-based model. It is free for up to 3 users, then $20/user/month. They claim this provides "unlimited traffic," but in reality, the limit is 10M requests per month, after which you need an enterprise plan.
Features, like flag scheduling, permissions, custom fields, and the visual editor are only available on the GrowthBook Pro paid plan.
Example scenarios
To give you an idea of what pricing looks like in reality, here are some example situations and their estimated costs for both PostHog and GrowthBook.
Note: GrowthBook does not display their Enterprise pricing needed to go beyond 10M requests per month.
Seats | Requests | PostHog cost | GrowthBook cost |
3 | 1,000,000 | $0 | $0 |
5 | 2,000,000 | $100 | $100 |
15 | 4,000,000 | $300 | $300 |
20 | 15,000,000 | $1400 | ??? (Enterprise) |
Notes:
- Using backend local evaluation in PostHog lowers the amount of flag usage depending on the polling duration and active number of servers. If you use locally evaluated flags with one server polling every 30 seconds, this amount is under 1M requests (free).
- PostHog has volume discounts on flags over 2 million requests per month.
Integrations
GrowthBook has integrations with data warehouses and analytics tools they rely on, but little beyond that. PostHog has a wider range of integrations.
PostHog | GrowthBook | |
Exports Export data to other sources | ✔ |