Confidence intervals and tests Updated November 2026

Statistical power

Statistical power is the probability that a test rejects the null hypothesis when the alternative is true: power = 1 - beta. It rises with sample size, effect size and alpha, and falls with variability. The usual target is 80%, which fixes beta at 0.2.

Syntax

n per group = 2 * (z(alpha/2) + z(beta))^2 * sigma^2 / delta^2   (two means)

Part	What it is
`z(alpha/2)`	1.96 for a two-sided test at alpha = 0.05
`z(beta)`	0.84 for 80% power, 1.28 for 90% power
`sigma`	the standard deviation of the outcome
`delta`	the smallest difference worth detecting

Examples

Two means: sigma = 10, delta = 5, alpha 0.05, power 0.8
n = 2 * (1.96 + 0.84)^2 * 100 / 25 = 2 * 7.84 * 4

Result: 63 per group (62.7 rounded up)

Two proportions: 10% vs 12%, alpha 0.05, power 0.8
n = (1.96 + 0.84)^2 * (0.09 + 0.1056) / 0.02^2 = 7.84 * 0.1956 / 0.0004

Result: about 3,834 per group

Raising power from 80% to 90% in the first example
2 * (1.96 + 1.28)^2 * 100 / 25

Result: 84 per group (about one third more data)

Common mistakes

Computing power after the study from the observed effect (post hoc power); it is just a restatement of the p-value and tells you nothing new.
Planning for an effect larger than is realistic so the sample looks affordable; the study ends up underpowered.
Confusing power with the significance level; alpha controls false positives, power controls false negatives.

Practice Statistical power

These drills check your answer on live data, free, no account needed.

Sample size for a proportionA pollster wants to estimate the share of residents who use public transit to within 2% (a margin of 0.02) with 90% confidence. There is no prior estimate, so use the conservative p = 0.5. Use z* = 1.645. How many residents must be surveyed? Give a whole number.
Sample size for a meanA planning study puts the standard deviation of customer spend at $34.00. How many customers must be sampled to estimate mean spend within $4.00 with 99% confidence? Use z* = 2.576. Give a whole number.

Questions

Why is 80% power the standard?

Convention, balancing cost against the 20% risk of missing a real effect. Clinical trials and expensive experiments often use 90%.

What is effect size?

The size of the difference you want to detect, often standardized as Cohen's d = delta / sigma. d = 0.2 is small, 0.5 medium, 0.8 large; a d of 0.5 needs about 64 per group at 80% power.

Can I increase power without more data?

A little: use a paired design, reduce measurement noise, or use a one-sided test when justified. Mostly, power comes from sample size.

Type I and Type II errorsA Type I error rejects a true null (false positive, probability alpha); a Type II error misses a false null (false negative, probability beta). Examples.
Sample sizeSample size formulas n = (z sigma / E)^2 for a mean and n = z^2 p(1 - p) / E^2 for a proportion, worked examples giving 97 and 385, and rounding rules.
A/B test significanceHow to tell whether an A/B test result is statistically significant: compute relative lift, run the two-proportion z-test, read the p-value and avoid peeking.
Hypothesis testing stepsThe five steps of a hypothesis test: state H0 and H1, choose alpha, compute the test statistic, find the p-value or critical value, decide. Worked example.