A/B tests help you make your Astro app better by enabling you to compare the impact of changes on key metrics. To show you how to set one up, we create a basic Astro app, add PostHog, create an A/B test, and implement the code for it.
1. Create an Astro app
First, ensure Node.js is installed (version 18.0 or newer). Then, create a new Astro app:
Terminal
npm create astro@latest
When prompted in the command line, name your new project directory (we chose astro-ab-test), start your new project Empty, choose No for TypeScript, install dependencies, and No for git repository.
Next, replace the code in src/pages/index.astro with a simple heading and button:
Once you’ve done this, reload your app and click the button a few times. You should see events appearing in the PostHog events explorer.
3. Capture a custom event
The first part of setting up our A/B test in PostHog is setting up the goal metric. We'll use the number of clicks on the button as our goal.
To measure this, we capture a custom eventhome_button_clicked when the button is clicked. To do this, update the code in posthog.astro to add a <script> and call posthog.capture() when the button is clicked.
With this set up, refresh your app and click the button a few times to see the event captured in PostHog.
4. Create an A/B test in PostHog
Next, go to the A/B testing tab and create an A/B test by clicking the New experiment button. Add the following details to your experiment:
Name it "My cool experiment".
Set "Feature flag key" to my-cool-experiment.
Use the default values for all other fields.
Click Save as draft.
Once created, set the primary metric to a trend of home_button_clicked and then click Launch.
5. Implement the A/B test code
When it comes to implementing our experiment code, there are two options:
Client-side rendering
Server-side rendering
We'll show you how to implement both.
Client-side rendering
To implement the A/B test, we use the posthog.onFeatureFlags callback to update the button text based on whether the user is in the control or test variant of the experiment.
Update the code in /components/posthog.astro to implement posthog.onFeatureFlags code inside PostHog's loaded callback:
Now if you refresh your app, you should see the button text updated to either Control variant or Test variant. Users are automatically split between the two, PostHog continues to track button clicks, and you can view the results of the A/B test in PostHog.
Server-side rendering
Notice that when you refresh the page, the button text flickers between Click me! and Control/Test variant. This is because it takes time for PostHog to load and make the feature flag request.
Server-side rendering is a way to avoid this. This fetches the feature flag before the page loads on the client.
To set this up, we must install and use PostHog’s Node library (because we are making server-side requests).
Terminal
npminstall posthog-node
In the src folder, create a posthog-node.js file. This is where we set up the code to create the PostHog Node client. You can find both your API key and instance address in your project settings.
Now, when you refresh the page, the button text is already set when the page loads.
Setting the correct distinctId
You may notice that we set distinctId = 'placeholder-user-id' in our flag call above. In production apps, to ensure you fetch the correct flag value for your user, distinctId should be set to their unique ID.
For logged-in users, you typically use their email as their distinctId. However, for logged-out users, you can use the distinct_id property from their PostHog cookie:
Note that Astro.request.headers is not available for static sites. If you want to access the request cookies, you need to set your output to server or hybrid in astro.config.mjs: