Linear regression

Correlation says two variables move together. Linear regression goes further: it draws the single best line through the cloud, turning a relationship into a prediction machine.

The line of best fit

Of all the lines you could draw through a scatter, regression picks the one that makes the residuals — the vertical gaps between the points and the line — as small as possible:

Precisely, it minimizes the sum of the squared residuals, which is why it's called least squares. The result is one line:

$\hat{y} = a + bx$

The hat on ŷ marks it as a prediction, not a measured value. The slope b says how much ŷ changes per one-unit rise in x; it's tied straight to the correlation:

$b = r \, \frac{s_y}{s_x}$

Reading the slope, and R²

Two numbers do most of the interpreting:

• The slope is the story in context: "each extra hour studied predicts about 4 more points."
• R² (just r squared) is the fraction of the variation in y that the line explains. R² = 0.7 means the line accounts for 70% of y's ups and downs; the rest is scatter.

The extrapolation trap

A regression line is only trustworthy within the range of the data it was fit on. Predict far outside that range and you're guessing: a line fit to a child's growth would "predict" a 20-foot-tall adult. Fit inside your data; be skeptical outside it.

Why this matters

Least-squares regression is the foundation the whole tower of predictive modeling is built on — from a spreadsheet trendline to the linear layer inside a neural network. And it caps this course: you began by summarizing one variable, and you end by modeling how one predicts another — the essence of learning from data.