Normal Distribution
nothing is called 95% until it has been counted
Concept

The line-up test — where it appears

A visual test in which the real data's plot is hidden among plots made from data simulated under the null, and a viewer is asked to pick it out. Its false-alarm rate is one over the number of panels, whatever the viewer looks for.

Named by 2 essays across 2 fields — each of them below, with the objects they name alongside it.

Twenty residual plots from data where the model is exactly right, n = 24. Every panel is a correctly specified linear model with normal errors. The apparent curvature, funnelling and outliers are all produced by noise, and the largest single residual across the twenty is 2.13 standard deviations of the error. This is the reference nobody has when judging a real residual plot.

Twenty residual plots

Judging whether a residual plot looks wrong requires knowing what a correct one looks like, and almost nobody has seen twenty of those. Here they are, from a model that is exactly right, at the sample size that matters.

regression · Qq How often each one-number summary of a quantile plot detects five departures from normality, forty observations. Power at 5% against five departures. The straightness statistic detects the skewed source 83.3% of the time and light tails 30.8%; kurtosis detects light tails 72.5%; skewness detects light tails 0.2%. Rejecting when any of the five exceeds its own 5% point rejects 13.9% of normal samples; correcting that search to 5% gives it 37.4% power against light tails.

A number for the shape

Five one-number summaries of a quantile plot's departure from its line, each a correct 5% test on forty normal observations, and no two agree about what matters. Straightness catches a skewed source 83% of the time and light tails 31%; kurtosis catches light tails 72% and skewness 0.2%. Reading all five and reporting whichever looks bad rejects 13.9% of genuinely normal samples. Correcting that search to 5% costs almost half the power against the departure it would have caught best — and a line-up of twenty panels gets the same correction for nothing.

lineup · Qq

Named alongside it

The objects these essays reach for when they reach for this one.

Visual inferenceError rateKurtosisModel diagnosticsMultiple comparisonsNormalityQ–Q plotResidual plotSample sizeSkewnessStatistical power

All concepts