Normal Distribution
nothing is called 95% until it has been counted
Concept

Convergence rate — where it appears

The power of the sample size at which a quantity shrinks, which distinguishes a better constant from a better procedure. Two procedures with the same rate differ by a constant that more data will not close, and two with different rates cross somewhere.

Named by 10 essays across 7 fields — each of them below, with the objects they name alongside it.

A pair pulled back at 20% of the gap per step. Above, the two series. Below, the difference between them. The gap is pulled back towards zero by 20% of itself each step, so it stays inside a band of 14.3 while the series themselves travel much further. Nothing here is stationary except the difference. The faint line below is the gap for two free walks from the same seed, drawn for comparison.

The regression that is not spurious

Two random walks regressed on each other are called significantly related three times in four, so the time-series field ends in a warning. The exception it names and does not measure is here — and when the pair is genuinely tied, the fitted relation converges at rate 1/n rather than the usual 1/√n.

cointegration · Spurious The normal approximation's error on a sum of 100 exponential draws, under its Berry–Esseen bound. The distance between the exact distribution function and the normal one peaks at 0.0133, at z = -0.01. The Berry–Esseen bound is 0.1146, 8.62 times the real worst error, and larger than the whole 2.5% tail a two-sided test reads.

A bound written for a coin

The Berry–Esseen theorem guarantees how far a standardised sum can be from the normal, and the guarantee is true. On an exponential source it is 8.62 times the real worst error at every sample size, the worst error sits at the centre rather than in a tail, and at a hundred draws the bound is larger than the 2.5% tail it would be asked to vouch for.

expansion · Rate Two rates, not a factor. The standard deviation of the covariate imbalance under three rules, at five trial sizes, 260 trials each, on log axes. The upper line is a coin: its slope is -0.489, against a closed form of exactly −½. The middle line is minimisation on a median split; its slope is -0.519 — the same rate — because inside a category the assignment is still a coin, and what it buys is the constant, 0.654 of a coin's at n = 200. The lower line is the rule that reads x and maximises the information about the treatment effect: slope -0.987, nearly twice as steep. Its advantage is therefore not a number that can be quoted — it is 0.258 of a coin's at n = 50 and 0.065 at n = 800, and it keeps going.

The rule that reads the number

Stop categorising and let the rule read the covariate itself. What it should minimise is not an invented distance but the variance of the effect being estimated — and what comes back is not a better constant but a different rate.

continuous · Assignment Where the normal approximation converges, and where it does not. Relative error against the exact binomial. At n = 1280 the error at the median is 0.96% and three sigma out it is 25.7% — a factor of 27. The tail is where the approximation is used.

The tail converges last

The central limit theorem is usually shown as a shape arriving. What the demonstration leaves out is the rate — and the rate is wildly different in the middle and in the tail, which is where every approximation in the subject is actually read.

normal · Rate One tail arrives; the other is still on its way at a million. The Kolmogorov distance between the exact law of a normalised maximum and its Gumbel limit, at six block sizes, for two parents that both have that same limit. Both are closed form: the exact law of a maximum is F(x)^n and no simulation is involved. The exponential parent's distance falls from 0.0280 to 2.707e-7 — a factor of a hundred thousand, which is exactly one over n. The normal parent's falls from 0.0522 only to 0.0091, a factor of 5.74, because its rate is one over log n. At a million readings a block the two differ by a factor of 33556.3.

The maximum converges slowly

The rate at which a normalised maximum reaches its limit law is computable rather than simulable, because the exact law of a maximum is always available. For a normal parent the distance falls like one over the logarithm of the block and is still 0.0091 at a million readings; for an exponential parent, with the same limit, it is 2.707×10⁻⁷.

extreme · Extremes The upper tail of 10 exponential draws: normal, one Edgeworth term, two Edgeworth terms, each against the exact tail. Each curve is an approximation divided by the exact gamma tail, so 1 is exact. Six standard deviations out at n = 10: the normal gives ×0.0000670, one Edgeworth term ×0.00159, two ×0.0167 and the saddlepoint ×1.0007.

A correction that goes below zero

One Edgeworth term takes the normal approximation's error at two standard deviations from 38% to 8% on ten exponential draws, and stretches the range within 10% of the truth from 1.66 to 3.09 standard deviations at a hundred. It also turns negative in the short tail at every sample size — past 3.13 standard deviations at a hundred draws and 9.83 at a hundred thousand — because the region recedes only as the sixth root of n.

expansion · Rate The squared estimate 1 standard errors from the flat point, exact and linearised. At δ = √n·μ/σ = 1 the exact law of the squared estimate has mean 2.00, variance 6.00 and skewness 2.177; the delta method's normal has mean 1.00, variance 4.00, no skewness, and 30.85% of its mass below zero, where a square cannot go. The Kolmogorov distance between them is 0.3085.

Where the derivative is zero

The delta method reads a standard error off a tangent line, and at a flat point the tangent says the spread is zero. The interval built on it for a squared mean covers 99.991% there and 85.978% one and a half standard errors away, with nearly every miss on the same side — and the law it should have used is a χ², not a normal.

normal · Clt The bounded error and the unbounded one. How the sequential trace procedure's answer is distributed, against the sample length, for a three-series system with 2 genuine relations. Over-counting — claiming a stationary combination that is a random walk — reads 4.9%, 7.2%, 5.7%, 6.2%, 5.9%, 4.2% across the six lengths, never far from the 5% of a single test. Under-counting reads 69.5%, 40.2%, 14.0%, 0.5%, 0.0%, 0.0%. The procedure is described as a 5% rule and the 5% applies to one of those columns.

The rank is a decision

The sequential procedure's 5% bounds one of its two errors. Over-counting reads between 4.2% and 7.2% at every sample length from fifty observations to three hundred; under-counting reads 69.5% at fifty and 0.0% at three hundred, and nothing in the procedure bounds it.

systems · Rank How fast a gap has to close before a sample can see it close. The power of the test against the half-life of a disagreement, at 100, 200, 400 observations, each read against its own simulated critical value. Every pair in every reading is genuinely tied together, so a non-rejection is a miss. At 200 observations a gap that halves in 3 steps is found 99.9% of the time and one that halves in 12 steps is found 15.3% of the time — and by 35 steps the reading is 6.1%, which is the test's own size. Beyond that the curves are flat because there is nothing left to detect with.

How slow a return a sample can see

At two hundred observations the test finds a gap that halves in five steps four times in five, one that halves in eight 37.3% of the time, and one that halves in fifty 4.95% of the time — which is the rate at which it finds pairs with no mechanism at all. The boundary moves with the sample, not with its square root.

timeseries · Spurious One of these converges and the other does not. Two measurements on the same fits, against the sample length, for a system with 2 genuine relations. The distance from the fitted plane to the true plane falls from 0.1438 at 100 observations to 0.0075 at 1600 — halving with each doubling, which is the 1/n rate this field's estimates converge at. The angle between the leading fitted relation and the leading generating one reads 29.6° and 29.0° at those same lengths, and is flat in between. The plane is an estimate; the relation inside it is not.

A space is not a relation

The fitted plane approaches the true one at rate 1/n — 0.1438 at a hundred observations and 0.0075 at sixteen hundred. The angle between the leading fitted relation and the leading generating one reads 29.6° and 29.0° at those same lengths, and never moves.

systems · Rank

Named alongside it

The objects these essays reach for when they reach for this one.

Central limit theoremCointegrationMonte CarloSkewnessTail probabilityClosed formNormal approximationRandom walkBerry–EsseenCointegrating rankCommon trendCritical value

All concepts