Back to the on-screen lesson ·
The slope and intercept that make the squared vertical misses as small as possible, why the line passes through the point of means, and what changing the data does to it.
Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.
You will compute a least squares slope and intercept from the two sums of deviations, evaluate the fitted line at a given explanatory value, plot it, and say exactly which quantity the fitting minimises. You will also predict what a shift or a rescaling of the data does to the fitted line without refitting it.
The last unit compared two groups defined in advance. Regression handles the case where the explanatory variable is a number rather than a label, and the question is how the response changes along it. The estimation ideas are the ones from unit one; the arithmetic is two sums.
The response is the variable being predicted and the explanatory variable is what predicts it; the choice between them is made by the question, not by the data. A fitted value is the height of the line at a given explanatory value. The point of means is the point whose coordinates are the two sample means. Least squares chooses the line minimising the sum of squared vertical misses.
Given pairs $(x_1, y_1), \dots, (x_n, y_n)$, the model is
$$Y_i = \beta_0 + \beta_1 x_i + \varepsilon_i,$$
and least squares chooses $\hat\beta_0$ and $\hat\beta_1$ to minimise $\sum (y_i - \beta_0 - \beta_1 x_i)^2$. Differentiating and setting both derivatives to zero gives
$$\hat\beta_1 = \frac{S_{xy}}{S_{xx}}, \qquad \hat\beta_0 = \bar y - \hat\beta_1 \bar x,$$
where $S_{xy} = \sum (x_i - \bar x)(y_i - \bar y)$ and $S_{xx} = \sum (x_i - \bar x)^2$.
Three features are worth reading off those formulas. The denominator involves $x$ alone, so a set of explanatory values that barely vary gives a slope with almost nothing to divide by — and a wildly unstable estimate. The intercept is not separately fitted: the second equation says the line passes through $(\bar x, \bar y)$, which least squares forces rather than assumes. And the misses are measured vertically, because the line is being used to predict $y$ from $x$; a line fitted the other way round is a different line.
Squaring rather than taking sizes is what makes the solution unique and computable in closed form. It is also what makes a single far-out point able to drag the whole line towards it, which is a genuine cost and the reason diagnostics exist.
Another way: picture
A scatter of points with a straight line through it, and a short vertical segment drawn from each point to the line. Least squares makes the total area of the squares built on those segments as small as it can be. Tilting or sliding the line makes some squares smaller and others larger, and the fitted line is where the total stops falling.
Another way: steps
| $x$ | $y$ | $x - \bar x$ | $y - \bar y$ | product | squared |
|---|---|---|---|---|---|
| 1 | 3 | -2 | -4 | 8 | 4 |
| 2 | 6 | -1 | -1 | 1 | 1 |
| 3 | 7 | 0 | 0 | 0 | 0 |
| 4 | 8 | 1 | 1 | 1 | 1 |
| 5 | 11 | 2 | 4 | 8 | 4 |
The means are $\bar x = 3$ and $\bar y = 7$; the two totals are $S_{xy} = 18$ and $S_{xx} = 10$. So the slope is $1.8$ and the intercept is $7 - 1.8 \times 3 = 1.6$. Notice the middle row: a point sitting exactly at the point of means contributes nothing to either total and therefore nothing to the line.
Dividing by $S_{yy}$. That gives a correlation, which is a different quantity with no units and answers a different question.
Fitting the intercept independently. It is determined by the slope and the point of means. There is one free choice, not two.
Measuring misses perpendicular to the line. That is a defensible fit for a question where neither variable is singled out, and it is not this one.
Swapping the two variables and expecting the reciprocal slope. Regressing $x$ on $y$ gives a different line through the same point of means, and its slope is not $1/\hat\beta_1$ unless the fit is perfect.
Reading the slope as what would happen if $x$ were changed. It describes how $y$ varies with $x$ in these data. Whether changing $x$ would change $y$ is a question about the design.
$S_{xy} = 100$, $S_{xx} = 40$, $\bar x = 5$, $\bar y = 30$.
Everything the formulas need.
Slope $100/40 = 2.5$.
Products over squares.
Intercept $30 - 2.5 \times 5 = 17.5$, so the line is $17.5 + 2.5x$.
Through the point of means.
The same $S_{xy}$ but with the explanatory values crowded together, so $S_{xx} = 2$.
A tiny denominator.
The slope becomes $50$, and moving one point a little changes it enormously.
Nothing to divide by.
A design that spreads the explanatory values out estimates the slope far better, from the same number of observations.
A design rule from a formula.
The slope is $36/12$.
Products over squares.
That is $3$.
The intercept is $20 - 3 \times 4 = 8$, so the line is $8 + 3x$.
For a set of points, the sum of products of the two deviations is $18$ and the sum of squared deviations of the explanatory variable is $6$. What is the least squares slope?
Answer:
For a set of points, the sum of products of the two deviations is $35$ and the sum of squared deviations of the explanatory variable is $7$. The two means are $5$ for the explanatory variable and $62$ for the response. Fill in the three lines of the fit.
| Value | |
|---|---|
| Slope | |
| Intercept | |
| Fitted value at an explanatory value of $10$ |
The fitted line is the intercept $2$ plus $2$ times the explanatory variable. Give the slope, and the fitted value when the explanatory variable is $9$.
Slope: s. Fitted value: f.
The fitted line is the intercept $4$ plus $3$ times the explanatory variable. Plot its height at explanatory values of $0$, $1$, $2$ and $3$.
Plot your answer on the grid:
A line has been fitted by least squares. Match each change to the data with its effect on the fitted line.
| The slope is unchanged; the intercept rises by $6$ | The slope and the intercept are both multiplied by $6$ | The slope is unchanged; the intercept falls by $6$ times the slope | Neither the slope nor the intercept changes | |
|---|---|---|---|---|
| Add $6$ to every response value | ||||
| Multiply every response value by $6$ | ||||
| Add $6$ to every explanatory value | ||||
| Add one more point, exactly at the point of the two means |
Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.
A line is fitted to $16$ points by least squares. Which quantity does the fitting make as small as possible?
You can fit a line from the two sums, find its intercept through the point of means, and name what least squares makes small. Say in your own words why the misses are measured vertically rather than perpendicular to the line.
9. Your turn: $S_{xy} = 36$, $S_{xx} = 12$, $\bar x = 4$, $\bar y = 20$, step 3