Website A/B Testing compares two variants of a page or element, with traffic split between them at random, measuring which achieves a better result. A meaningful test requires a sufficient volume of visits and conversions – with only a few dozen conversions a month, a test will never reach a reliable result, and deciding on that basis is worse than not deciding at all. The test should run for full weeks so it covers weekends too, and should not be stopped early at the first favourable number. It is worth testing changes with potentially large impact, not button colour. With low traffic, other methods are more suitable – usability testing, customer interviews and removing obvious obstacles in the process.
See also: Statistical significance of a test, Multivariate testing, Hypothesis in CRO.
