Normal Distribution
nothing is called 95% until it has been counted
Concept

Variance inflation — where it appears

How much a coefficient's variance is multiplied by when a design loses its orthogonality, whether through a correlated predictor or a lost run. For an orthogonal design losing one run it is exactly one plus the reciprocal of the spare runs, which makes it a property of the plan rather than of the accident.

Named by 7 essays across 5 fields — each of them below, with the objects they name alongside it.

Twenty series with a lag-one correlation of 0.8. Every series has a true mean of zero and 60 observations. The marks on the right are the twenty sample means. The variance of that mean is 8.3 times what 60 independent observations would give, so the series is worth about 7 of them.

The observations that repeat each other

Almost every standard error divides by √n, which claims the observations carry independent information. At a lag-one correlation of 0.8 a fifty-point series is worth about six independent observations, and its 95% interval covers 47%.

timeseries · Dependence The damage and the warning, against the same dial. Two readings at each persistence. In the darker colour, how often a regression between two independent series of 200 steps is called significant at 5%: 4.9% at φ = 0, 34.2% at 0.8, 52.4% at 0.9, 83.4% at a unit root. In the lighter, how often the standard unit-root test refuses a unit root on one of those series — the chance the analyst is told the series is stationary and may be regressed: 87.2% at φ = 0.9 and 31.9% at 0.95. At φ = 0.9 both are high at once, which is a correct diagnostic licensing a regression that is wrong half the time.

The cliff that is a slope

A regression between two independent series is called significant 4.9% of the time at no persistence, 52.4% at a lag-one correlation of 0.9, and 83.4% at a unit root. The rule the field offers asks whether the last of those holds, and at 0.9 the unit-root test correctly refuses one 87.2% of the time.

timeseries · Spurious A penalty that counts independent rows. Regret of five selection rules over fifteen candidates as the design and the errors are made persistent at the same coefficient, 300 draws at each of 7 settings, scored on fresh rows. Akaike's criterion gives up 0.0193, 0.0196, 0.0294, 0.0428, 0.0666, 0.1075, 0.1281; charging each coefficient its own variance inflation, with the persistences read off the sample, gives up 0.0188, 0.0231, 0.0339, 0.0375, 0.0496, 0.0656, 0.0652, and with them given 0.0193, 0.0236, 0.0348, 0.0374, 0.0426, 0.0469, 0.0318. The hold-out gives up 0.0358, 0.0365, 0.0393, 0.0453, 0.0594, 0.0913, 0.1087 and Schwarz's criterion 0.0355, 0.0295, 0.0331, 0.0397, 0.0575, 0.0849, 0.1047.

A penalty that counts independent rows

Akaike's penalty charges a fit two error variances a coefficient because each coefficient is fitted from n independent rows. When the design and the errors both persist, a row is worth less than a row, and the honest charge is twice the coefficient's variance inflation. On a hundred and twenty rows at persistence 0.7 that charge is 34.7 against an optimism counted at 34.5, where Akaike's charges 10. Charged it, the criterion's regret at 0.95 falls from 0.128 to 0.032 with the persistences given and to 0.065 when they are read off the sample — and at 0.5, where the charge is right and Akaike's is half of it, Akaike's still selects better.

proxy · Forecast A point that decides a coefficient, on two covariates correlated at 0.9. A hundred points on two normal covariates with correlation 0.9. Dashed lines mark where the most extreme five per cent of each covariate begin. The point carrying the largest share of the first coefficient's information, 14.2%, is inside the lines on both axes, so neither covariate's plot on its own would show it. The coefficient rests on 26.1 effective points, with a variance inflation of 6.27.

A point ordinary on every axis

In a multiple regression each coefficient rests on its own number of points, n divided by the kurtosis of that covariate's residual after the others, exactly. Correlation between normal covariates inflates a coefficient's variance fivefold and leaves it resting on the same third of the points. What correlation does instead is hide the point that decides the coefficient: at a correlation of 0.9 it stands out on no single covariate's plot in 77.3% of designs of a hundred, and the largest leverage a normal design reaches depends on the number of points and covariates and on nothing else.

spread · Summary What one lost run costs a 16-run factorial fitting 11 coefficients. Every run is worth the same: dropping any one multiplies every coefficient's variance by 1.2000, which is 1 + 1/(N − p) with N = 16 and p = 11, and gives every pair of coefficients a correlation of 0.1667 where the complete design had exactly zero.

The run that did not happen

Lose one run from any orthogonal design and every coefficient's variance is multiplied by exactly 1 + 1/(N − p), and every pair of coefficients acquires a correlation of exactly 1/(N − p + 1) where there was none. The price is set by the design's spare capacity and by nothing else, and a saturated design cannot survive it at all.

design · Factorial A point that decides an interaction, on two covariates correlated at 0. A hundred points on two normal covariates with correlation 0. Dashed lines mark where the most extreme five per cent of each covariate begin. The point carrying the largest share of the interaction's information, 10.2%, is inside the lines on both axes, so neither covariate's plot on its own would show it. The coefficient rests on 19.0 effective points.

A term built from the others

A regression's squared terms and interactions are columns built from its covariates, and each coefficient on one rests on the number of points divided by the kurtosis of what the built column adds — fifteen for the square of a normal covariate, nine for the product of two uncorrelated ones, three for a treatment's interaction with a normal covariate, which is the same as the covariate's own coefficient. A design of a hundred rests on more than the population says, 12.2 points behind a square against 6.7, because a hundred draws rarely hold the tail that sets the kurtosis. Centring changes nothing about the square's coefficient, to the last digit. And correlation between two covariates hides the point that decides each one's own coefficient while it reveals the point that decides their interaction.

spread · Summary The shape one real modifier leaves on a table of eight characteristics, and one trial's table. Expected interaction z when characteristic 1 modifies the effect at h = 1: 2.80, 2.00, 2.00, 2.00, 0.00, 0.00, 0.00, 0.00 — the real row 2.80, its three nested neighbours 0.713 of that, the four unrelated characteristics nothing. One seeded trial of 400 reads 1.91, 1.90, 2.06, 1.95, 0.14, 0.33, 0.39, 2.08.

The pattern a cause leaves on its neighbours

A characteristic that modifies a treatment's effect raises every characteristic that overlaps it, in proportion to the overlap, and fitting that pattern was supposed to say which one is the cause. Fitted with the noise's own correlation it names the largest row on every trial, because the noise draws the same pattern — so the 54.4% at which the largest row is right among nested characteristics is the most any reading of the table can manage, and the honest answer is a set of about five.

alongside · Repetition

Named alongside it

The objects these essays reach for when they reach for this one.

Effective sample sizeAutocorrelationCorrelationExperimental designLeast squaresDependenceInteractionKurtosisLeverageModel diagnosticsMonte CarloStationarity

All concepts