A/B testing set-up
A/B tests show which version of a page wins more enquiries or sales. We set up the testing platform, tracking and analysis.
Prerequisites
Testing pays off when a page has enough traffic and a change could measurably affect enquiries or revenue. Larger changes can first be tried on a share of visitors. With little traffic, user testing or before-and-after comparisons work better.
Tools
Google Optimize was discontinued in 2023, and no free default tool has replaced it. Depending on requirements, we use VWO, Kameleoon, Optimizely or the open-source tool GrowthBook.
Server-side tests avoid the flash of the original version in the browser and leave load time untouched. In return, each test needs development effort.
How a test runs
Hypothesis: What changes, why, and which metric should improve.
Planning: Calculate sample size and duration in advance.
Implementation: Build variants, check tracking, start the test.
Analysis: Only after the planned duration, with attention to significance and segments.
Common mistakes
Stopping a test early because an interim result looks good
Too little traffic for small effects
Testing several changes at once
Durations that do not cover full weeks
Data protection
Testing platforms set cookies or store identifiers, so they generally need consent. We connect them to your consent management and prefer providers that process data in the EU.