Normal Distribution
nothing is called 95% until it has been counted
Concept

Influence — where it appears

How much a fitted result changes when one observation is removed, which combines where the observation sits with how far it falls from the fit. It is measured by refitting without the row, and two rows standing together can each look uninfluential while jointly deciding the answer.

Named by 5 essays across 3 fields — each of them below, with the objects they name alongside it.

Twenty points and one more, at leverage 0.74. Without the distant point the slope is 0.495; with it the slope is -0.389. Its leverage is 0.737 and its Cook's distance is 24.1, against a conventional threshold of 1.

The line that one point drew

A single observation among twenty-one reverses the sign of a fitted relationship. Its leverage is known from its x value before the outcome is looked at, so this is a property of the design rather than a surprise in the data.

regression · Leverage What a design does to the residuals of a correct model. Every residual has standard deviation sigma times the square root of one minus its leverage. On this design the leverages run from 0.045 to 0.663, so the residual spreads differ by a factor of 1.68 — and the model is exactly right. The high-leverage point's residual averages 0.46 of the fitted spread where a typical point's averages 0.79.

Residuals are not the errors

A residual's standard deviation is σ√(1 − hᵢᵢ), so a design whose leverages run from 0.045 to 0.663 produces residuals whose spreads differ by a factor of 1.68 with the model exactly right. On the samples where the high-leverage point really did have the largest error, a raw residual plot shows it as the largest on 0.0% of them.

lineup · Qq A discrepancy of 4 σ at the top of the range: one bad run, or a line that bends there, in two designs of twelve runs. The same twelve errors in every panel. With one run at the top (upper panels) a bad observation and a bent line produce identical data, point for point. With three runs at the top (lower panels) the bad observation leaves one run far from its two neighbours and the bent line moves all three together. The dashed line is the true line; the grey line is the least-squares fit.

Three runs at the end of the line

With one run at the top of a regression's range, a bad observation there and a line that bends there produce data with exactly the same distribution, so no residual, influence measure or test can say which happened. Move six of twelve runs to the ends, three at each, and a discrepancy of four σ is named correctly as a bad run 84.1% of the time and as a bend 91.6% — while the slope's standard error falls from 0.0836 to 0.0709. The design, not the diagnostic, decides whether the question has an answer.

lineup · Qq Two far rows, and the line with one of them deleted. Twenty clean points and two rows near x = 9. The slope is −0.511 with every row, −0.376 with one far row deleted, and 0.495 with both deleted. Deleting one of them barely moves the line, because the other is still there.

Two points that hide each other

One far observation among twenty-one has a Cook's distance of 24.1. Put a second beside it and the two read 0.966 and 0.772, neither crossing 1, while together they reverse the slope and deleting both moves the fit by 53.3.

regression · Leverage How many of 100 points a fitted slope rests on, by the distribution the covariate was drawn from. The effective number of points behind a slope is n divided by the design's kurtosis. Over designs of 100, its median share is two points, half at each: 1.000; uniform: 0.551; normal: 0.349; exponential: 0.160; Student's t, 3 df: 0.148; lognormal, σ = 1: 0.076 — 34.9 of 100 points for a normal covariate and 7.6 for a lognormal one.

The points a slope rests on

Before any outcome is measured, a regression's design already says how many of its points the slope will rest on: n divided by the covariate's kurtosis, exactly. A hundred points of a normal covariate carry a slope on about 35 of them; a hundred of a lognormal one, on 7.6. The largest leverage follows the same tail — above one half in 15.7% of lognormal designs of a hundred and in none of the normal ones — and on a correct straight line, deleting the single highest-leverage point moves the slope by more than its standard error in 16.3% of lognormal datasets of a hundred and 0.1% of normal ones.

spread · Summary

Named alongside it

The objects these essays reach for when they reach for this one.

LeverageModel diagnosticsExperimental designLeast squaresCook's distanceHat matrixOutliersResidual plotStudentised residualBonferroniClosed formEffective sample size

All concepts