Analyze Experiments with PostHog AI

PostHog AI creates Experiments, analyzes results, and summarizes behavioral differences between variants. You can set up A/B tests, check statistical significance, and compare session replays across variants – all through natural language.

How it works

When you ask PostHog AI to create an Experiment, it sets up the Experiment with your specified split percentages and goal metrics, creates the feature flag with the right variants, and configures targeting rules if you specify a user segment.

For existing Experiments, PostHog AI can summarize results, interpret statistical significance, and explain what the data means in plain language.

Analyze results with PostHog AI

PostHog AI doesn't just show you the numbers, it interprets them. When you ask about Experiment results, it tells you:

• Which variant is winning and by how much
• Whether the result is statistically significant based on the confidence intervals
• What behavioral differences exist between variants by summarizing session replays for each group
• What to do next – whether to ship, extend, or stop the Experiment

Try it

Select a prompt to try it out in the PostHog app:

• Set up an A/B test with a 70/30 split for a new checkout flow
• Summarize the results of my latest Experiment
• What behavioral differences do you see between variant A and variant B?
• Summarize session replays for the control group in my pricing Experiment
• Create an Experiment targeting only mobile users

Tips for better results

• Name your goal metric – "with conversion on purchase_completed" helps PostHog AI configure the right success metric
• Specify split percentages – "70/30 split" or "equal split across three variants" tells PostHog AI exactly how to allocate traffic
• Ask about significance – "Is this result statistically significant?" prompts PostHog AI to interpret the confidence intervals
• Combine with session summaries – "Summarize replays for users in the control group" reveals behavioral differences that metrics alone might miss
• Ask for recommendations – "Should I ship this?" prompts PostHog AI to weigh the evidence and suggest next steps

Get started

To start analyzing Experiments with PostHog AI, set up PostHog AI.