Level 4: Inference · Problem 69 of 87

A/B test: sample size per group

A statistics problem from Statisticsdrill. Solve it with one answer box, with a fresh sample on every attempt and the worked steps and spreadsheet function in the answer, and it is checked the moment you submit.

The task

You plan an A/B test on a page that converts at 4% and want to detect a rise to 5% with a two-sided test at α = 0.05 and 90% power. Use n = (z_α/2 + z_β)² × [p1(1 - p1) + p2(1 - p2)] / (p2 - p1)², with z_α/2 = 1.96 and z_β = 1.28. How many visitors are needed in each group? Round up. Give a whole number.

What this problem teaches

Halving the difference you want to detect roughly quadruples the sample size.

A hint if you get stuck

Add the two z values and square the sum, multiply by the sum of the two variances p(1 - p), and divide by the squared difference. Always round a sample size up.