Concept

Clopper–Pearson — where it appears

An interval for a proportion made by inverting two exact one-sided binomial tests. Its coverage never falls below the stated level at any proportion, which it pays for with width and with an average coverage above the level.

Named by 10 essays across 4 fields — each of them below, with the objects they name alongside it.

Worst and average coverage of six 95% intervals for a proportion, 30 trials. The worst coverage over every proportion beside the average over a uniform one, with the average expected width. Clopper–Pearson: worst 95.05%, average 97.34%, width 0.299. Blaker: worst 95.00%, average 96.31%, width 0.283. Wilson: worst 83.71%, average 95.24%, width 0.271.

What a guaranteed minimum costs

Clopper–Pearson's interval never covers less than 95%, and at thirty trials it averages 97.34% and is 10.4% wider than Wilson's. Blaker's interval keeps the same guarantee, averages 96.31% and is 4.6% wider. The difference is not waste: Clopper–Pearson guarantees each side separately, holding both below 2.5%, and Blaker guarantees only their sum — so at ten trials and a proportion of 0.15 it misses on one side 5.00% of the time.

discrete · Oscillation
Clopper–Pearson, mid-p and the randomised interval: coverage across the proportion, 20 trials. Clopper–Pearson never falls below 95% and runs up to 99.80%. The mid-p interval, which is the randomised interval with its coin fixed at one half, runs from 92.94% to 99.80%. The randomised interval covers 95% at every proportion, to within the 0.043% of the numerical integration over the coin.

The coin that makes it exact

Every interval for a proportion either covers less than 95% somewhere or more than 95% on average, because a count is discrete. One construction covers exactly 95% at every proportion: it adds a uniform random draw to the count. At thirty trials it is 0.9% wider than Wilson's interval and narrower than both exact ones — and two analysts with the same data report different intervals, and one study in forty that sees nothing reports an empty one.

discrete · Oscillation
Expected width against coverage, n = 30, p = 0.15. The Wald interval is the shortest and covers 94.2%. Clopper–Pearson covers 98.3% and is 13% wider. Shortness is not a virtue on its own — an interval of zero width is the shortest of all.

The shortest interval is the one that misses

Four intervals for the same data, with their widths and their coverage measured together. The narrowest is the one that fails its stated level, which is exactly why it looks the most appealing.

intervals · Width
Thirty surveys of a thousand people, true prevalence 0.1%, test 90% sensitive and 95% specific. Each row is one survey. The thin bar is the corrected estimate ± 1.96 standard errors; the heavy bar is the exact interval for the positive rate carried through the correction; the tick is the one-sided 95% upper limit. 30 of the thirty printed intervals reach below zero and 0 lie entirely below it.

An upper limit is the finding

A survey of a thousand people with a 90%-sensitive, 95%-specific test, estimating a prevalence of one in a thousand, prints a corrected 95% interval that covers the truth 95.17% of the time — and reaches below zero in 97.95% of surveys and lies entirely below zero in 3.35%. Coverage is not the problem. The problem is what the interval can say, and the honest summary of a typical survey is one number: the prevalence is below 1.64%. Bringing that limit down to twice the truth takes 184,521 people at this specificity and 5,083 with a test that never gives a false positive.

screening · Baserate
Coverage across the proportion of the randomised interval with the arrangement of the successes as its coin, 20 trials. Between proportions of 0.2 and 0.8 the arrangement coin's coverage stays within 0.121% of 95%, where mid-p's strays by 2.76%. Its worst coverage is 92.60%, at a proportion of 0.140, near the ends of the range where the count leaves few arrangements; mid-p's is 92.94%. Averaged over the proportion, 95.63% against mid-p's 96.11% and Clopper–Pearson's 97.71%.

The coin that is already there

The randomised interval covers exactly 95% at every proportion because it adds a drawn coin to the count, and a drawn coin is why nobody reports it. A sequence of trials already holds one: given the count, the order the successes arrived in is equally likely to be any of its arrangements whatever the proportion is. Used as the coin, it keeps twenty trials' coverage within 0.12 points of 95% from proportions of 0.2 to 0.8, where mid-p strays by 2.8, and it gives the same interval to every analyst. It fails where the count has few arrangements — a count of none has one — and it holds only while the order is read as recorded: an analyst choosing among sixteen orders lifts a 5% test on sixteen trials to 7.68%.

discrete · Oscillation
Coverage against the expected count of the interval whose coin is read from the arrival times. The randomised interval for a Poisson mean, with its coin read from where in the window the events fell, covers exactly 97.5% at every expected count between 0.025 and ln 40 = 3.689 and exactly 95% past it, when a count of zero is given the exact interval's [0, ln 40]; its worst is 95.00%. Given mid-p's [0, ln 20] instead, it falls to 92.50% just past 3.00. Mid-p itself reaches 91.66% at 3.03; the exact interval never falls below 95.62% and averages well above 95%.

The coin a clock supplies

A count observed over a window comes with the times its events arrived, and given the count those times are uniform whatever the rate is — so they are a coin with a continuum of faces at every count from one up. Used to randomise the interval for a Poisson mean, they leave exactly one count without a coin, the count of zero, and what that count is given decides everything: mid-p's limit of ln 20 leaves a hole at 92.5%, and the exact interval's ln 40 is the shortest limit that leaves none, after which the interval covers exactly 97.5% below an expected count of 3.69 and exactly 95% above it.

discrete · Oscillation
A leak read off a cohort, at two cuts. The threshold rule's leak as a cohort of 1,000 units estimates it from its own two-by-two table: the bar is the middle eighty per cent of 600 cohorts' plug-in estimates, the open circle their median and the stroke the true leak, for five copulas at a Spearman rank correlation of 0.4. At the 90th percentile the estimate is centred on the truth and its middle eighty per cent spans about fourteen points: Gaussian 12.8% to 26.4% around 19.2%, t on four 21.5% to 35.7% around 28.9%, Frank 5.4% to 18.7% around 12.2%, Clayton 0.8% to 9.6% around 4.4%, Clayton turned over 35.9% to 49.5% around 43.1%. At the 99th it is not centred on anything: Gaussian 13.1% to 100.0% around 13.7%, t on four 17.5% to 72.9% around 37.4%, Frank 14.4% to 100.0% around 2.2%, Clayton 16.1% to 100.0% around 0.6%, Clayton turned over 37.0% to 75.7% around 56.3%, because on three of the five copulas between two cohorts in five and four in five have nobody past both cuts.

A leak read off a two-by-two table

The share of an interaction a threshold-balancing rule leaves alone needs one number from the data: how often a unit past the cut on one covariate is past it on the other. A cohort of a thousand estimates it to within a dozen points at the 90th percentile. At the 99th most cohorts have an empty cell in their table, and every one of those reads a leak of exactly one hundred per cent — a reading the bootstrap then confirms with an interval of zero width.

copula · Criterion
Coverage of an interval for a ratio of two rates against the expected total. The unconditional coverage of three intervals for the ratio of two Poisson rates, each built conditionally on the total and each reporting the whole of (0, ∞) when the total is zero, at the conditional proportion 0.5, against the expected total count. With the arrival times in both windows as the coin the interval covers exactly 95% plus a twentieth of e raised to minus the expected total: 96.84% at an expected total of one, 95.25% at three, 95.00% from about ten. The exact conditional interval covers 100.00%, 99.79%, 97.82% and 96.55% at forty; mid-p 99.11% at three and 95.89% at ten.

Two windows and a ratio

Two counts from two windows give an interval for the ratio of their rates through the conditional binomial: given the total, the first window's count is binomial with a proportion that depends only on the ratio. The arrival times in both windows supply that binomial with a continuous coin at every total from one up, and the interval then covers exactly 95% plus a twentieth of the chance of no events at all. A total of zero can only report the whole of (0, ∞). And the exact conditional test's familiar conservatism is not the zero total's: the zero is the larger cause only while the expected total is below ln 2, and at an expected total of ten the exact interval still covers 97.8% for reasons that have nothing to do with it.

discrete · Oscillation
Three promises, and no procedure keeps all three. Average coverage and worst-case coverage for four 95% intervals for a proportion at n = 40, computed exactly. Their expected widths are 0.2418, 0.2417, 0.2472, 0.2641 in the same order. The textbook interval and the score interval have the same expected width to four digits — 0.2418 and 0.2417 — and worst-case coverages of 55.31% and 92.21%. The exact interval never breaks its promise and is 9.3% wider than the score interval to do it. Each of the three columns orders the four procedures differently.

An interval that covers and says nothing

A procedure returning the whole line 95% of the time and the empty set otherwise has coverage exactly 95% at every parameter value. Two real intervals at forty observations have expected widths of 0.2418 and 0.2417 and worst-case coverages of 55.31% and 92.21%.

intervals · Coverage
The textbook (Wald) interval's coverage among studies whose count of 40 lands in the tails. Coverage of the nominal 95% textbook (Wald) interval at 40 trials, conditional on the count being at most k or at least 40 − k, computed exactly over the counts for every true proportion. With k = 2 its highest value anywhere is 88.96%; With k = 5 its highest value anywhere is 94.90%; With k = 8 its highest value anywhere is 95.57%. A set whose highest value stays below 95% is a set of counts on which the label is wrong whatever the proportion is.

A count that bets against its interval

A reader who sees only the count can bet against an interval's 95%, and win whatever the proportion is, exactly when some set of counts carries less than 95% at every proportion. Out of forty trials the textbook interval has one — a count of five or fewer, or thirty-five or more, covers at most 94.90% — and the score interval has none: a prior exists under which every one of its forty-one intervals is 96.46% credible, and that prior is the proof that no bet can be found.

intervals · Coverage

Named alongside it

The objects these essays reach for when they reach for this one.

CoverageDiscretenessConservative intervalInterval widthWilson intervalMid-pRandomised intervalBinomial proportionConfidence intervalExchangeabilityReproducibilityWald interval

All concepts