Residuals are not the errors

Twenty residual plots calibrates the eye against a model that is exactly right. The panels are drawn from a design whose points are evenly spread, and the calibration is honest for that design.

Change the design and the calibration stops being honest, for a reason that has nothing to do with the model.

Why they are different objects #

The model’s errors ε are what the equation says: independent, mean zero, and all with the same standard deviation σ. That is the assumption a residual plot is drawn to check.

The residuals are what is left after fitting, and fitting uses up part of each error. The algebra gives

$$\operatorname{sd}(e_i) = \sigma\sqrt{1 - h_{ii}},$$

with $h_{ii}$ the leverage of point i — $1/n + (x_i - \bar{x})^2/S_{xx}$ for a simple regression, a quantity that depends only on the x values and can be computed before any y is looked at.

So residuals are not identically distributed, and the amount by which they are not is set by the design. On the design drawn above the leverages run from 0.045 to 0.663, so the residual standard deviations run from 0.977σ down to 0.581σ — a factor of 1.68, with the model exactly right and nothing wrong anywhere.

A quantile plot of raw residuals is therefore reading a sample of differently-distributed values against a line that assumes they are the same. It is heteroskedastic by construction, and the construction is the design rather than the data.

The word heteroskedastic is worth pausing on, because it is what a residual plot is most often drawn to look for. A plot showing residuals that fan out with the fitted value is read as evidence that the model’s errors have non-constant variance — a real defect with real consequences for every standard error in the output. What the algebra above says is that some of the fanning is produced by the design and would be there under a perfectly homoskedastic model.

So the diagnostic and the artefact have the same signature, and the only way to tell them apart is to compute the leverages and see whether the pattern in the residuals matches the pattern in $\sqrt{1 - h_{ii}}$. Standardising does that subtraction automatically, which is a second reason to do it and is rarely the reason given.

The point it hides is the point that matters #

The consequence is not that the plot is slightly miscalibrated. It is that the miscalibration is aimed at exactly the wrong observation.

Leverage shrinks the residual and raises the influence. A high-leverage point pulls the fitted line towards itself, so the line passes close to it, so its residual is small — and a point that moves the line is the point whose error does the most damage. The two effects are the same effect, and they push the diagnostic and the danger in opposite directions.

The measurement makes it concrete. On the clumped design, the far point’s error was the largest error the model made on 3.4% of samples. Of those samples:

The raw residual plot never once showed it. Its residual is systematically 0.58 of what an error of the same size would produce, and that is enough to put it behind the clump’s ordinary points every time.

And it happens on ordinary designs too #

The clumped design is extreme, chosen to make the mechanism unmistakable. The effect is present at much milder designs and the loss is already substantial.

On an evenly spread design — the one everybody’s intuition is built on — the raw plot finds the influential point’s error 37% of the time and standardising raises it to 47%. A quarter of the available detections are being lost on the best-behaved design there is.

Two further readings sit in that table.

Neither plot finds it often. Even standardised, the detection rate is under half on the friendly designs — which is the same weakness a normality band has seen from the other side, because the largest error and the largest residual are different things on any sample where a second point happens to be unlucky. A residual plot is a weak instrument for identifying which observation the model got most wrong, and standardising improves a weak instrument rather than repairing it.

The gap widens as the design worsens. At three quarters clumped, standardising nearly doubles the detection rate. That is where the repair is worth most, and it is also where a reader is least likely to think about it, because a clumped design looks like a data-collection accident rather than a statistical decision.

The two constraints, and where they go #

The residuals are not merely differently scaled; they are dependent, and the dependence has an exact form worth knowing because it explains where the missing variance went.

Fitting an intercept and a slope imposes two linear constraints: the residuals sum to zero and their inner product with x is zero. Two constraints on n numbers leaves n − 2 free, which is the degrees of freedom, and it is also why Σ$h_{ii}$ = 2 exactly for a simple regression.

That last identity is the accounting. The leverages sum to the number of parameters, so a design that gives one point a leverage of 0.663 has taken that from the other twenty-three — and their leverages are correspondingly smaller, their residuals correspondingly closer to the full σ. Leverage is conserved, and concentrating it on one point is the same act as removing it from the rest.

The consequence is visible in the table’s first column. On the evenly spread design the leverages run from 0.042 to 0.183 and sum to 2; on the clumped one from 0.045 to 0.663 and also sum to 2. Nothing was added by clumping; one point took most of what there was.

That framing makes the design criterion easier to state. A design’s diagnosability is how evenly it spreads its two units of leverage, and an even spread is 2/n each — 0.083 at twenty-four points, which is very nearly what the evenly spread design achieves.

What standardising does, and what it does not #

Dividing each residual by its own estimated standard deviation — $e_i/\bigl(s\sqrt{1 - h_{ii}}\bigr)$ — produces the studentised residual, and it repairs exactly one thing.

It fixes the spreads. Every studentised residual has the same variance by construction, so a quantile plot of them is reading a sample that really is identically distributed. That is the whole of the repair and it is not small: it is the difference between the two columns above.

It does not fix the dependence. Residuals are correlated with each other — they have to be, since they satisfy two linear constraints — and studentising does nothing about that. At n = 24 with two parameters fitted, the correlations are small; at n = 6 they are not, and a quantile plot of six residuals is reading a sample whose points are noticeably tied together.

It does not fix the masking. A studentised residual is computed with the point included in the fit, so a genuinely bad point still drags the line towards itself and still shrinks its own residual — less than before, and not to zero. The deleted residual, computed from a fit that excludes the point, is the version that removes this, and two bad points can still hide each other from any one-at-a-time deletion.

So the ordering of the three residuals is a sequence of repairs: raw fixes nothing, studentised fixes the spreads, deleted fixes the spreads and the self-pull, and none of them fixes a pair.

The version of this that everybody knows, and the version nobody does #

Two facts about leverage are standard and one is not, which is worth separating because they are the same fact.

Standard: a high-leverage point can drag the line. That is the point that drew the line, it is taught with the fourth dataset of the quartet — where no summary and no dependence measure reaches it — and every diagnostic textbook has a version of it.

Standard: Cook’s distance combines residual and leverage. The influence of a point is roughly its squared residual times h/(1 − h)², so a point with both a large residual and high leverage is flagged.

Not standard: the residual in that formula is systematically small for exactly the points whose leverage is large. Cook’s distance handles it — the h/(1 − h)² factor grows fast enough to compensate — but the plot does not, and the plot is what gets looked at first. A reader scanning a residual plot before computing any influence measure is scanning a display in which the influential points have been quietly shrunk.

So the practice that works is the one that does not start with the plot: compute the leverages from the design, note which points are high, and read the plot with those points marked. The leverages need no outcome data at all, so this can be done before the experiment runs — which turns the diagnostic from something applied afterwards into something the design is checked against.

The part that is a design decision #

Everything above is decided before a single outcome is measured, which makes it a design question rather than an analysis one.

The leverages are a function of the x values. A design that spreads its points evenly has leverages near 2/n and a residual plot that is very nearly honest. A design with a clump and an outlier has one leverage near 1 and a residual plot that cannot see the point that matters.

That gives a design criterion nobody states in these terms. A design’s maximum leverage is a bound on how diagnosable it is, and a design chosen for precision — which puts its points at the extremes to maximise the spread of x — is buying precision with diagnosability. An optimal design for estimating a slope is a design in which a bad observation at either end is undetectable.

The trade is real and it is usually made without being noticed. A middling design has a slightly larger standard error for the slope and a residual plot that works; the optimal one has the best possible standard error conditional on the model being right, and no way of finding out that it is not.

Where the missing variance went, in one number #

The accounting can be closed exactly, which is the tidiest way to hold the whole page.

The sum of the residuals’ variances is $\sigma^2\sum(1 - h_{ii}) = \sigma^2(n - p)$, with p the number of parameters. The sum of the errors’ variances is $\sigma^2 n$. So fitting removes exactly $p$ units of $\sigma^2$ from the residuals, one per parameter, and the degrees of freedom is that removal counted.

What the design decides is whose variance is removed. An even design takes p/n from each point; a clumped design takes most of it from one. The total is the same and the distribution is not, and it is the distribution that decides what a plot can see.

That is also why the effect vanishes as n grows with the design held even: p/n goes to zero, every leverage goes to zero, and the residuals converge on the errors. It does not vanish for a design whose maximum leverage stays put — adding more points to the clump while leaving the far point where it is leaves that point’s leverage almost unchanged, because it is still the only point out there.

So the rule is not “this matters at small n”. It is this matters when one point carries a large share of the design, which is a condition on the shape of the x values and not on how many of them there are.

What this does to a normality check on residuals #

The essays before this one are about a quantile plot of the data, and in a regression the plot is of residuals, so the two effects compound.

The residuals are not identically distributed, which is this page. A quantile plot of raw residuals from a clumped design has a built-in S-shape, because the low-spread points sit closer to the line than a normal sample’s order statistics would.

The residuals are not independent, which the two constraints guarantee. A quantile plot’s reference distribution assumes independence, and residuals from a small sample are noticeably tied.

And the plot is answering a question about the errors’ shape, which the regression’s own inference mostly does not need — the coefficients are averages and inherit the limit theorem in the same way a sample mean does.

So a residual quantile plot carries two construction artefacts and answers a question of limited relevance. The repair for the first is studentising, for the second is nothing much at realistic sample sizes, and for the third is to be clear about which inference the plot is being consulted about: a prediction interval for a new observation needs the errors’ shape and a confidence interval for a slope largely does not.

None of that makes the plot useless. It makes the plot’s mild departures uninformative, which is exactly the region a reader spends most time interpreting.

What is claimed here, and what is not #

Two statements, and both are about the design rather than about any sample.

The design’s leverages are not all equal. That is the premise, it is computable from the x values alone before any outcome exists, and equal leverages would mean the calculation had been replaced by 2/n.

The high-leverage point’s residual is smaller on the raw plot than on the standardised one — 0.46 against 0.79 in units of the fitted spread, which is the ratio $\sqrt{1 - h}$ the algebra predicts and which is measured rather than assumed.

The reading that does not survive is raw residuals taken as the model’s errors. That reading needs the residual standard deviations of a correct model to be equal, and they differ by a factor of 1.68 on a model that is exactly right. No property of the data produced that: the design did, before the data existed.

Still open: what an honest residual plot of a bad design would look like #

Standardising equalises the spreads and the table above says it recovers only part of what was lost. The rest is the information that was never in the residual to begin with.

At a leverage of 0.663, the point’s residual carries 34% of its error’s variance and the fit absorbed the other 66%. Rescaling recovers the scale and cannot recover the information: the data simply do not say much about whether that point is wrong, because the line went where the point told it to.

What would say is a comparison against something the point did not influence — a fit excluding it, a prediction from the rest of the data, a replicate at the same x. The first is the deleted residual and is standard; the third is a design decision, and it is the one that actually solves the problem. A design with replication at its extreme points can tell a bad observation from a wrong model, and a design without it cannot, whatever residual is plotted.

Still open: what a diagnosable design costs in precision #

What is not measured here is the exchange rate: how much replication at the ends costs in precision, and how much detection it buys. That is a two-dimensional sweep over the design space with a diagnostic’s power on one axis and the slope’s standard error on the other, and it would turn an argument about diagnosability into the same kind of priced trade as everything else here.

The shape of the answer is guessable and the size is not. Moving a few points off the extremes costs precision quadratically in how far they move — the slope’s standard error depends on the spread of x — and buys diagnosability roughly linearly, since it is the maximum leverage that matters and that falls in proportion. So a small retreat from the optimal design should be cheap and worth taking, which is the standard advice about not using a two-point design, arriving here with a second reason attached. How small is the measurement, and an optimal design’s own field has the arithmetic for it.