Field

What a diagnostic plot is showing

A quantile plot is read by eye against a band drawn for one point at a time, and a genuinely normal sample of forty has forty chances to leave it: 45.0% of them do. The band a reader is actually using is that one widened by a factor of 1.502, and nothing draws it. The quantity is wrong as well as the width — a t interval needs the sampling distribution of the mean to be normal rather than the data, and the source that leaves the band on every sample of forty covers 94.80% while one that leaves it on 57% covers 95.73%. And what is plotted are not the errors: a design whose leverages run from 0.045 to 0.663 gives residuals whose spreads differ by a factor of 1.68 with the model exactly right.

Two 95% bands for a quantile plot of 40 points. The outer band is left by 5% of genuinely normal samples — which is what a reader is using a band for. The inner one holds each point separately at 95%, which is what software draws, and 45.0% of genuinely normal samples step outside it. The outer is the inner widened by a factor of 1.502.

The band the eye was standing in for

The confidence band software draws on a quantile plot holds each point at 95%, and a genuinely normal sample of forty has forty chances to leave it — so 45.0% of them do. The band a reader is actually using is that one widened by a factor of 1.502, and nothing draws it.

What the plot says, and what the interval does, at n = 40. For each source: how often a quantile plot of the data leaves its pointwise band, and how often the 95% t interval for the mean misses. The two-lump source leaves the band on 100% of samples and its interval covers 94.80%; the t on three degrees of freedom leaves it on 57% and covers 95.73%, the best of the five.

The plot is about the wrong quantity

A t interval needs the sampling distribution of the mean to be normal, not the data. A two-lump source leaves its quantile band on 100% of samples of forty and its interval covers 94.80%; a t on three degrees of freedom leaves it on 57% and covers 95.73%, the best of five sources.

What a design does to the residuals of a correct model. Every residual has standard deviation sigma times the square root of one minus its leverage. On this design the leverages run from 0.045 to 0.663, so the residual spreads differ by a factor of 1.68 — and the model is exactly right. The high-leverage point's residual averages 0.46 of the fitted spread where a typical point's averages 0.79.

Residuals are not the errors

A residual's standard deviation is σ√(1 − hᵢᵢ), so a design whose leverages run from 0.045 to 0.663 produces residuals whose spreads differ by a factor of 1.68 with the model exactly right. On the samples where the high-leverage point really did have the largest error, a raw residual plot shows it as the largest on 0.0% of them.

All essays