Back to the on-screen lesson ·
Splitting every observation into the part the line accounts for and the part it does not, and the constraints the fitting imposes on what is left.
Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.
You will compute fitted values and residuals from a fitted line, plot residuals against the explanatory variable, and use the fact that they add to zero as an arithmetic check and as the source of the degrees of freedom a regression has. You will also say why that fact is no evidence about the quality of the fit.
The last lesson produced a line. This one looks at what the line did not capture, one observation at a time. Everything needed is the equation of the fitted line and a subtraction — and the subtraction has to go in the right order.
The fitted value $\hat y_i$ is the height of the line at $x_i$. The residual $e_i = y_i - \hat y_i$ is the observed response minus the fitted value: positive above the line, negative below it. A residual plot puts $x$ along the bottom and $e$ up the side, so the fitted line becomes the horizontal axis.
For each point the fit produces
$$\hat y_i = \hat\beta_0 + \hat\beta_1 x_i, \qquad e_i = y_i - \hat y_i,$$
and therefore $y_i = \hat y_i + e_i$: every observation is the part the line accounts for plus the part it does not. The order of the subtraction is a convention and it is universal — observed minus fitted — so that a point above the line has a positive residual.
Two identities hold for every least squares fit with an intercept, whatever the data:
$$\sum_{i=1}^{n} e_i = 0, \qquad \sum_{i=1}^{n} x_i e_i = 0.$$
The first comes from minimising over the intercept, the second from minimising over the slope. Both are consequences of the fitting, not findings about the data, so neither is any kind of evidence that the line is appropriate. A hopeless line has residuals that add to zero too.
They do have a real consequence: $n$ residuals from a line with an intercept and a slope satisfy two constraints, so they carry only $n - 2$ independent pieces of information. That is where the divisor in the residual variance comes from, and it is the same degrees-of-freedom bookkeeping as the $n - 1$ in the sample variance.
The residual plot is the point of all this. Against a sloping line, a bend or a widening spread is hard to see; against a flat axis it is obvious. Two lessons from here that plot decides whether the fit should be believed at all.
Another way: picture
The same scatter twice. On the left, points around a sloping line. On the right, the line has been flattened into the horizontal axis and each point moved down by the height of the line beneath it. The pattern that was hidden by the slope — a bend, a fan — is unmistakable in the second picture and invisible in the first.
Another way: steps
The line $\hat y = 1.6 + 1.8x$, fitted to five points.
| $x$ | $y$ | Fitted | Residual |
|---|---|---|---|
| 1 | 3 | 3.4 | -0.4 |
| 2 | 6 | 5.2 | 0.8 |
| 3 | 7 | 7.0 | 0.0 |
| 4 | 8 | 8.8 | -0.8 |
| 5 | 11 | 10.6 | 0.4 |
The residuals add to zero exactly, as they must. They also alternate in sign rather than drifting, which is what a plot would show as a structureless band — the picture a fit is supposed to produce. Had they run negative, positive, positive, negative, the line would be missing a bend.
Subtracting fitted minus observed. The sign of every residual is then reversed, and so is every conclusion drawn from the plot.
Reading the residuals adding to zero as evidence of a good fit. It is forced by the fitting and holds for the worst line in the world.
Reading a residual as an error. The error $\varepsilon_i$ is the unobservable departure from the true line; the residual is the observable departure from the fitted one. They are close when the fit is good and they are not the same object.
Plotting residuals against the response. The response contains the fitted value, so that plot shows a relationship by construction. Plot against the explanatory variable or against the fitted value.
The line is $\hat y = 5 + 3x$, and at $x = 4$ the response was $14$.
Line first.
Fitted value $5 + 12 = 17$.
The height of the line.
Residual $14 - 17 = -3$: the point lies three below the line.
Observed minus fitted.
Four residuals are computed as $2.5$, $-1.5$, $-3$ and $2$.
From four observations.
They add to $0$, so the arithmetic is consistent with a fit that had an intercept.
A check, not a diagnostic.
Had they added to $0.7$, a slip has been made somewhere — in a fitted value, or in the line itself.
Worth doing every time.
The fitted value is $10 + 12$.
Evaluate the line.
That is $22$.
So the residual is $20 - 22 = -2$.
The fitted line is the intercept $5$ plus $5$ times the explanatory variable. Three observations were taken at explanatory values $1$, $2$ and $3$, with responses $15$, $15$ and $15$. Give the fitted value and the residual for each.
| Fitted value | Residual | |
|---|---|---|
| At an explanatory value of $1$ | ||
| At an explanatory value of $2$ | ||
| At an explanatory value of $3$ |
The fitted line is the intercept $2$ plus $4$ times the explanatory variable. At an explanatory value of $6$ the observed response was $19$. What is the residual?
Answer:
A line has been fitted to four observations, at explanatory values $1$, $2$, $3$ and $4$. The observations fell $2$ above the line, $3$ below it, $3$ above it and $2$ below it, in that order. Plot the four residuals against the explanatory values.
Plot your answer on the grid:
The fitted line is the intercept $6$ plus $5$ times the explanatory variable. At an explanatory value of $6$ the observed response was $37$. Give the fitted value and the residual.
Fitted value: f. Residual: e.
A line with an intercept is fitted to four points. Three of the residuals are $3$, $-3$ and $4$. Put the marker at the fourth.
-15 |——————————| 15
Mark the position with a cross, then write the value:
Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.
A line with an intercept is fitted to $30$ points by least squares. Which statement is true of the resulting residuals, whatever the data look like?
You can compute a fitted value and a residual in the right order, draw a residual plot, and use the zero sum as a check. Say in your own words why residuals adding to zero says nothing about whether the line is appropriate.
9. Your turn: the line $\hat y = 10 + 2x$, and at $x = 6$ the response was $20$, step 3