Normal Distribution
nothing is called 95% until it has been counted
Concept

Studentised residual — where it appears

A residual divided by its own estimated standard deviation, s√(1 − h), so residuals at different leverages share one scale. The deleted version estimates s without the row itself, which stops a row inflating its own yardstick and does not stop a partner doing it.

Named by 5 essays across 5 fields — each of them below, with the objects they name alongside it.

What a design does to the residuals of a correct model. Every residual has standard deviation sigma times the square root of one minus its leverage. On this design the leverages run from 0.045 to 0.663, so the residual spreads differ by a factor of 1.68 — and the model is exactly right. The high-leverage point's residual averages 0.46 of the fitted spread where a typical point's averages 0.79.

Residuals are not the errors

A residual's standard deviation is σ√(1 − hᵢᵢ), so a design whose leverages run from 0.045 to 0.663 produces residuals whose spreads differ by a factor of 1.68 with the model exactly right. On the samples where the high-leverage point really did have the largest error, a raw residual plot shows it as the largest on 0.0% of them.

lineup · Qq What each correction is worth, exactly. Each variance estimate's expectation under a constant error variance, divided by the variance the slope actually has, at 20 rows on an even design, by two routes: the closed form E[eᵢ²] = σ²(1 − hᵢᵢ) carried through each correction's own weight, and the mean of 20000 counted estimates. The maximum leverage here is 0.1857 and the design's fourth-moment share Σu⁴/(Σu²)² is 0.0897, which is the only thing the closed form reads. HC0 comes out at 0.8603 — short by construction, since its factor is exactly 1 − 1/n − Σu⁴/(Σu²)². HC1 reaches 0.9559, HC2 is exactly 1.0000 at every design and every sample size, and HC3 overshoots to 1.1647. On an even design the four are within a fifth of each other and the choice barely matters.

Three corrections and a leverage

On an even design of twenty rows the four robust corrections read 0.8603, 0.9559, 1.0000 and 1.1647 of the truth and the choice barely matters. Add one point at x = 8 and they read 0.3191, 0.3419, 1.0000 and 5.1127.

sandwich · Misspecification Three summaries of a fitted line's geometry on Anscombe's four datasets, with each rule's threshold. Largest leverage: 0.318, 0.318, 0.318, 1.000 against a threshold of 0.364. Largest Cook's distance: 0.489, 0.808, 1.393, undefined, the point fits itself against 1 and against 0.364. Largest studentised deleted residual: 2.1, 2.2, 1192.2, undefined against a Bonferroni threshold of 3.90.

Two numbers for the fit's geometry

No summary of the dependence between x and y can flag Anscombe's third and fourth datasets. A summary of the fit's own geometry can, and it takes two numbers of different kinds: the largest leverage, which reads the design before any y is seen and flags the fourth with certainty, and the largest deleted residual against a Bonferroni threshold, which flags the third at 1,192 and raises a false alarm on 4.78% of honest datasets. The rule of thumb most often quoted, Cook's distance over 4/n, flags the honest line too — and 66.45% of honest datasets like it.

spread · Summary Two far rows, and the line with one of them deleted. Twenty clean points and two rows near x = 9. The slope is −0.511 with every row, −0.376 with one far row deleted, and 0.495 with both deleted. Deleting one of them barely moves the line, because the other is still there.

Two points that hide each other

One far observation among twenty-one has a Cook's distance of 24.1. Put a second beside it and the two read 0.966 and 0.772, neither crossing 1, while together they reverse the slope and deleting both moves the fit by 53.3.

regression · Leverage One statistic that is right under both hypotheses. Rejection rates for both statistics under both nulls, at 25% of 150 units treated, with the weak-null readings taken at an effect spread of 3. The difference in means is exact under the sharp null and rejects 22.93% of true weak nulls. The studentised difference is exact under the sharp null — 4.07% — and reads 6.27% under the weak one. The repair is a change of statistic inside the same construction: the same re-randomisations, the same fixed outcomes, a different number compared across them.

A statistic that is exact twice

Dividing the difference in means by its own separate-variance standard error before permuting takes the rejection rate under a true weak null from 20.47% to 6.07%, keeps the exactness under the sharp null at 4.07%, and costs 0.8 points of power against a real effect. At an even split it changes nothing at all, in every draw.

exact · Nuisance

Named alongside it

The objects these essays reach for when they reach for this one.

LeverageClosed formModel diagnosticsBonferroniCook's distanceHat matrixInfluenceLeast squaresLeave-one-outMonte CarloOutliersAllocation ratio

All concepts