Normal Distribution
nothing is called 95% until it has been counted
Series

Oscillation — the series

7 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. Coverage against sample size, true proportion 0.15. Coverage does not improve monotonically. n = 19 covers 93.8% while the larger n = 20 covers 81.9%. The sample space is discrete, so the endpoints jump as n changes.

    More data is not monotonically better

    Coverage of an interval for a proportion does not improve smoothly as the sample grows. It oscillates, and there are larger samples that cover materially worse than smaller ones — a sample of twenty covers twelve points worse than a sample of nineteen.

    part 2 · intervals
  2. Coverage of the Wilson interval against the expected count, at 10, 30, 100 and 1,000 trials. Read against the expected number of successes the four sample sizes draw the same curve near the boundary. The worst coverage is 83.50% at n = 10, 83.71% at n = 30, 83.79% at n = 100, 83.81% at n = 1000, each at an expected count near 0.177, and the limiting depth is e^(−0.1765) = 83.82%.

    A hole no sample size fills

    Wilson's interval is the recommended repair for a proportion, and away from the boundary it wobbles a point or two around 95%. Near zero it has a hole: at an expected count of 0.1765 its coverage is 83.50% at ten trials, 83.79% at a hundred and 83.81% at a thousand, and it never climbs past e to the minus 0.1765, which is 83.82%. The hole is where the interval built on one success stops containing the truth, and it belongs to the count rather than to the sample size.

    part 3 · discrete
  3. Worst and average coverage of six 95% intervals for a proportion, 30 trials. The worst coverage over every proportion beside the average over a uniform one, with the average expected width. Clopper–Pearson: worst 95.05%, average 97.34%, width 0.299. Blaker: worst 95.00%, average 96.31%, width 0.283. Wilson: worst 83.71%, average 95.24%, width 0.271.

    What a guaranteed minimum costs

    Clopper–Pearson's interval never covers less than 95%, and at thirty trials it averages 97.34% and is 10.4% wider than Wilson's. Blaker's interval keeps the same guarantee, averages 96.31% and is 4.6% wider. The difference is not waste: Clopper–Pearson guarantees each side separately, holding both below 2.5%, and Blaker guarantees only their sum — so at ten trials and a proportion of 0.15 it misses on one side 5.00% of the time.

    part 4 · discrete
  4. Clopper–Pearson, mid-p and the randomised interval: coverage across the proportion, 20 trials. Clopper–Pearson never falls below 95% and runs up to 99.80%. The mid-p interval, which is the randomised interval with its coin fixed at one half, runs from 92.94% to 99.80%. The randomised interval covers 95% at every proportion, to within the 0.043% of the numerical integration over the coin.

    The coin that makes it exact

    Every interval for a proportion either covers less than 95% somewhere or more than 95% on average, because a count is discrete. One construction covers exactly 95% at every proportion: it adds a uniform random draw to the count. At thirty trials it is 0.9% wider than Wilson's interval and narrower than both exact ones — and two analysts with the same data report different intervals, and one study in forty that sees nothing reports an empty one.

    part 5 · discrete
  5. Coverage across the proportion of the randomised interval with the arrangement of the successes as its coin, 20 trials. Between proportions of 0.2 and 0.8 the arrangement coin's coverage stays within 0.121% of 95%, where mid-p's strays by 2.76%. Its worst coverage is 92.60%, at a proportion of 0.140, near the ends of the range where the count leaves few arrangements; mid-p's is 92.94%. Averaged over the proportion, 95.63% against mid-p's 96.11% and Clopper–Pearson's 97.71%.

    The coin that is already there

    The randomised interval covers exactly 95% at every proportion because it adds a drawn coin to the count, and a drawn coin is why nobody reports it. A sequence of trials already holds one: given the count, the order the successes arrived in is equally likely to be any of its arrangements whatever the proportion is. Used as the coin, it keeps twenty trials' coverage within 0.12 points of 95% from proportions of 0.2 to 0.8, where mid-p strays by 2.8, and it gives the same interval to every analyst. It fails where the count has few arrangements — a count of none has one — and it holds only while the order is read as recorded: an analyst choosing among sixteen orders lifts a 5% test on sixteen trials to 7.68%.

    part 6 · discrete
  6. Coverage against the expected count of the interval whose coin is read from the arrival times. The randomised interval for a Poisson mean, with its coin read from where in the window the events fell, covers exactly 97.5% at every expected count between 0.025 and ln 40 = 3.689 and exactly 95% past it, when a count of zero is given the exact interval's [0, ln 40]; its worst is 95.00%. Given mid-p's [0, ln 20] instead, it falls to 92.50% just past 3.00. Mid-p itself reaches 91.66% at 3.03; the exact interval never falls below 95.62% and averages well above 95%.

    The coin a clock supplies

    A count observed over a window comes with the times its events arrived, and given the count those times are uniform whatever the rate is — so they are a coin with a continuum of faces at every count from one up. Used to randomise the interval for a Poisson mean, they leave exactly one count without a coin, the count of zero, and what that count is given decides everything: mid-p's limit of ln 20 leaves a hole at 92.5%, and the exact interval's ln 40 is the shortest limit that leaves none, after which the interval covers exactly 97.5% below an expected count of 3.69 and exactly 95% above it.

    part 7 · discrete
  7. Coverage of an interval for a ratio of two rates against the expected total. The unconditional coverage of three intervals for the ratio of two Poisson rates, each built conditionally on the total and each reporting the whole of (0, ∞) when the total is zero, at the conditional proportion 0.5, against the expected total count. With the arrival times in both windows as the coin the interval covers exactly 95% plus a twentieth of e raised to minus the expected total: 96.84% at an expected total of one, 95.25% at three, 95.00% from about ten. The exact conditional interval covers 100.00%, 99.79%, 97.82% and 96.55% at forty; mid-p 99.11% at three and 95.89% at ten.

    Two windows and a ratio

    Two counts from two windows give an interval for the ratio of their rates through the conditional binomial: given the total, the first window's count is binomial with a proportion that depends only on the ratio. The arrival times in both windows supply that binomial with a continuous coin at every total from one up, and the interval then covers exactly 95% plus a twentieth of the chance of no events at all. A total of zero can only report the whole of (0, ∞). And the exact conditional test's familiar conservatism is not the zero total's: the zero is the larger cause only while the expected total is below ln 2, and at an expected total of ten the exact interval still covers 97.8% for reasons that have nothing to do with it.

    part 8 · discrete

All series