Linear regression
Simple linear regression fits the line y = a + b * x that minimizes the squared vertical distances to the points. The slope is b = Sxy / Sxx (cross deviations over squared x deviations), the intercept is a = y-bar - b * x-bar, so the line passes through the point of means.
Syntax
y-hat = a + b * x, b = Sxy / Sxx, a = y-bar - b * x-bar| Part | What it is |
|---|---|
b | the slope: the change in y for a one-unit increase in x |
a | the intercept: the predicted y when x = 0 |
Sxy | sum of (x - x-bar)(y - y-bar) |
Sxx | sum of (x - x-bar)^2 |
Examples
x: 1, 2, 3, 4, 5 (x-bar = 3); y: 2, 4, 5, 4, 5 (y-bar = 4)
Sxy = 6, Sxx = 10
b = 6 / 10 = 0.6; a = 4 - 0.6 * 3Result: y-hat = 2.2 + 0.6x
Excel: =SLOPE(B1:B5, A1:A5) and =INTERCEPT(B1:B5, A1:A5); y comes first
Predict y at x = 6
2.2 + 0.6 * 6Result: 5.8
Excel: =FORECAST.LINEAR(6, B1:B5, A1:A5)
Interpret the slope: ad spend x in $1,000s, sales y in units, b = 0.6Result: each extra $1,000 of ad spend is associated with 0.6 more units sold, on average
Slope from correlation: b = r * sy / sx
0.775 * 1.225 / 1.581Result: 0.6
Common mistakes
- Swapping x and y in Excel; SLOPE, INTERCEPT and FORECAST.LINEAR take known_y first, then known_x.
- Predicting far outside the range of x (extrapolation); the line is only supported where you have data.
- Interpreting the intercept when x = 0 is impossible or far from the data; it is then just a fitting constant.
- Reading the slope as a causal effect; regression on observational data shows association, not cause.
Practice Linear regression
These drills check your answer on live data, free, no account needed.
- Regression slopeData on monthly ad spend and sales at regional stores (x = Ad spend ($k), y = Sales ($k)): (9, 87.8), (10, 98.7), (6, 81.3), (12, 99.1), (15, 111.9), (6, 78.6). What is the slope of the least squares line predicting y from x? Round to 3 decimal places.
- Regression interceptData on monthly ad spend and sales at regional stores (x = Ad spend ($k), y = Sales ($k)): (9, 87.8), (10, 98.7), (6, 81.3), (12, 99.1), (15, 111.9), (6, 78.6). What is the intercept of the least squares line predicting y from x? Round to 2 decimal places.
- Predicted value from a fitted lineData on monthly ad spend and sales at regional stores (x = Ad spend ($k), y = Sales ($k)): (9, 87.8), (10, 98.7), (6, 81.3), (12, 99.1), (15, 111.9), (6, 78.6). Fit the least squares line and predict y when x = 11. Round to 2 decimal places.
Questions
What does least squares mean?
The line is chosen so that the sum of squared residuals, the vertical gaps between points and line, is as small as possible. No other straight line has a smaller sum.
How do I run a regression in Excel?
For one predictor use SLOPE, INTERCEPT, RSQ and FORECAST.LINEAR, or LINEST for the full set of statistics. For several predictors use LINEST or the Regression tool in the Analysis ToolPak.
What are the assumptions of linear regression?
A linear relationship, independent observations, residuals with constant spread (homoscedasticity) and roughly normal residuals. Check them with a residual plot.
Related
- Correlation coefficient (Pearson r)How to calculate the Pearson correlation coefficient r by hand and with Excel CORREL, a worked example with exact numbers and what values of r mean.
- R-squaredR-squared is the share of variation in y explained by the regression, 1 - SSE / SST. Worked example with exact numbers, link to r, Excel RSQ and its limits.
- ResidualsA residual is observed y minus predicted y. How to compute residuals from a fitted line, a worked example, and residual plot patterns that signal a bad model.
- CovarianceHow to calculate sample and population covariance, a worked example with exact numbers, how covariance becomes correlation, and Excel COVARIANCE.S and .P.
Previous: Correlation coefficient (Pearson r)Next: R-squared