Normal Distribution
nothing is called 95% until it has been counted
Concept

Wilson interval — where it appears

An interval for a proportion made by inverting the score test rather than plugging the estimate into its own standard error. It covers close to its level on average, and near zero its coverage has a dip that no sample size removes.

Named by 7 essays across 2 fields — each of them below, with the objects they name alongside it.

Coverage of four nominal 95% intervals, n = 20. Computed exactly by summing over all 21 possible counts, not simulated. The Wald interval drops to 18.2% and is jagged everywhere; Clopper–Pearson never falls below 95% and pays for it in width.

What the 95% refers to

An interval that claims 95% is making a checkable statement about a procedure, not about the interval in front of you. Build every possible sample and count, and the interval taught first turns out to cover 87.6% of the time.

intervals · Coverage Coverage of the Wilson interval against the expected count, at 10, 30, 100 and 1,000 trials. Read against the expected number of successes the four sample sizes draw the same curve near the boundary. The worst coverage is 83.50% at n = 10, 83.71% at n = 30, 83.79% at n = 100, 83.81% at n = 1000, each at an expected count near 0.177, and the limiting depth is e^(−0.1765) = 83.82%.

A hole no sample size fills

Wilson's interval is the recommended repair for a proportion, and away from the boundary it wobbles a point or two around 95%. Near zero it has a hole: at an expected count of 0.1765 its coverage is 83.50% at ten trials, 83.79% at a hundred and 83.81% at a thousand, and it never climbs past e to the minus 0.1765, which is 83.82%. The hole is where the interval built on one success stops containing the truth, and it belongs to the count rather than to the sample size.

discrete · Oscillation Worst and average coverage of six 95% intervals for a proportion, 30 trials. The worst coverage over every proportion beside the average over a uniform one, with the average expected width. Clopper–Pearson: worst 95.05%, average 97.34%, width 0.299. Blaker: worst 95.00%, average 96.31%, width 0.283. Wilson: worst 83.71%, average 95.24%, width 0.271.

What a guaranteed minimum costs

Clopper–Pearson's interval never covers less than 95%, and at thirty trials it averages 97.34% and is 10.4% wider than Wilson's. Blaker's interval keeps the same guarantee, averages 96.31% and is 4.6% wider. The difference is not waste: Clopper–Pearson guarantees each side separately, holding both below 2.5%, and Blaker guarantees only their sum — so at ten trials and a proportion of 0.15 it misses on one side 5.00% of the time.

discrete · Oscillation Clopper–Pearson, mid-p and the randomised interval: coverage across the proportion, 20 trials. Clopper–Pearson never falls below 95% and runs up to 99.80%. The mid-p interval, which is the randomised interval with its coin fixed at one half, runs from 92.94% to 99.80%. The randomised interval covers 95% at every proportion, to within the 0.043% of the numerical integration over the coin.

The coin that makes it exact

Every interval for a proportion either covers less than 95% somewhere or more than 95% on average, because a count is discrete. One construction covers exactly 95% at every proportion: it adds a uniform random draw to the count. At thirty trials it is 0.9% wider than Wilson's interval and narrower than both exact ones — and two analysts with the same data report different intervals, and one study in forty that sees nothing reports an empty one.

discrete · Oscillation Coverage against the expected count of the interval whose coin is read from the arrival times. The randomised interval for a Poisson mean, with its coin read from where in the window the events fell, covers exactly 97.5% at every expected count between 0.025 and ln 40 = 3.689 and exactly 95% past it, when a count of zero is given the exact interval's [0, ln 40]; its worst is 95.00%. Given mid-p's [0, ln 20] instead, it falls to 92.50% just past 3.00. Mid-p itself reaches 91.66% at 3.03; the exact interval never falls below 95.62% and averages well above 95%.

The coin a clock supplies

A count observed over a window comes with the times its events arrived, and given the count those times are uniform whatever the rate is — so they are a coin with a continuum of faces at every count from one up. Used to randomise the interval for a Poisson mean, they leave exactly one count without a coin, the count of zero, and what that count is given decides everything: mid-p's limit of ln 20 leaves a hole at 92.5%, and the exact interval's ln 40 is the shortest limit that leaves none, after which the interval covers exactly 97.5% below an expected count of 3.69 and exactly 95% above it.

discrete · Oscillation Three promises, and no procedure keeps all three. Average coverage and worst-case coverage for four 95% intervals for a proportion at n = 40, computed exactly. Their expected widths are 0.2418, 0.2417, 0.2472, 0.2641 in the same order. The textbook interval and the score interval have the same expected width to four digits — 0.2418 and 0.2417 — and worst-case coverages of 55.31% and 92.21%. The exact interval never breaks its promise and is 9.3% wider than the score interval to do it. Each of the three columns orders the four procedures differently.

An interval that covers and says nothing

A procedure returning the whole line 95% of the time and the empty set otherwise has coverage exactly 95% at every parameter value. Two real intervals at forty observations have expected widths of 0.2418 and 0.2417 and worst-case coverages of 55.31% and 92.21%.

intervals · Coverage The textbook (Wald) interval's coverage among studies whose count of 40 lands in the tails. Coverage of the nominal 95% textbook (Wald) interval at 40 trials, conditional on the count being at most k or at least 40 − k, computed exactly over the counts for every true proportion. With k = 2 its highest value anywhere is 88.96%; With k = 5 its highest value anywhere is 94.90%; With k = 8 its highest value anywhere is 95.57%. A set whose highest value stays below 95% is a set of counts on which the label is wrong whatever the proportion is.

A count that bets against its interval

A reader who sees only the count can bet against an interval's 95%, and win whatever the proportion is, exactly when some set of counts carries less than 95% at every proportion. Out of forty trials the textbook interval has one — a count of five or fewer, or thirty-five or more, covers at most 94.90% — and the score interval has none: a prior exists under which every one of its forty-one intervals is 96.46% credible, and that prior is the proof that no bet can be found.

intervals · Coverage

Named alongside it

The objects these essays reach for when they reach for this one.

CoverageClopper–PearsonBinomial proportionDiscretenessConfidence intervalConservative intervalInterval widthWald intervalCredible intervalExact enumerationFrequentist interpretationMid-p

All concepts