Normal Distribution
nothing is called 95% until it has been counted
Concept

Prior predictive — where it appears

The distribution of data or of a test result averaged over a prior on the parameters, before anything is observed. Averaged power under it is the chance a planned study succeeds, which cannot exceed the prior chance that the effect is real.

Named by 3 essays across 2 fields — each of them below, with the objects they name alongside it.

The chance a trial succeeds against its size, when the expected effect of 0.5 is uncertain by four amounts. With the effect known, 80% is reached at 63 per arm. With the effect uncertain by 0.25 standard deviations it takes 113; by 0.5, 1268; by 0.75, no sample size at all, because the chance can never exceed the 74.8% prior probability that the effect is positive.

The chance a trial succeeds

A trial of sixty-four per arm has 80% power at an effect of half a standard deviation. If the effect is only believed to be about half a standard deviation, give or take a quarter, the chance the trial reaches significance is 69.2%; give or take a half, 61.4%. Reaching 80% then takes 113 per arm, or 1,268 — and when the belief is uncertain by three quarters of a standard deviation no number of patients reaches 80%, because the chance can never exceed the 74.8% probability that the effect is positive at all.

planned · Power A prior worth 35 observations, moved across the range — truth 0.1, n = 20. The same prior weight centred at each of 33 places. Its interval covers 100.0% where the centre is near the truth and 0.0% at its worst, while the mean width where it covers least is 0.221 against a flat prior's 0.263 on the same data.

When the prior is confident and wrong

A prior worth thirty-five observations, centred in the wrong place, produces a 95% interval that covers nothing at all — and reports a width 5% narrower than an honest one. It takes seventeen thousand observations to repair, not thirty-five, and the worst study to run is the one whose sample size equals the prior's weight, exactly.

bayes · Credible A prior centred on the truth's population and too confident about it: coverage across the population, and how often the check flags it. Truths drawn from a population of proportions centred at 0.2 and worth ten observations; twenty trials per study; the analyst's prior centred at 0.2 with the weight on the axis. Coverage of the 95% credible interval, exact over the population: 94.7% at 2, 96.0% at 5, 95.0% at 10, 88.1% at 20, 75.9% at 35, 61.3% at 60, 48.2% at 100, 34.0% at 200, 23.9% at 400. Share of studies whose count the prior-predictive check puts in its 5% tail: 0.1%, 1.8%, 3.2%, 14.2%, 17.3%, 17.3%, 21.9%, 21.9%, 21.9%.

A prior wrong about its own spread

A prior can be centred exactly where the truths lie and still be wrong about how far they spread. Ten times too confident — worth a hundred observations about proportions whose population is worth ten — its 95% credible intervals cover 48.2% of the population's studies at twenty trials, and nothing in the interval shows it: it is 0.456 of a flat prior's width, against 0.436 for a prior whose hundred observations' confidence is earned. A prior with tails too thin misses the one study in ten that comes from somewhere unusual a third of the time, and a flat component of two per cent brings that study's coverage from 65.6% to 84.0%.

bayes · Credible

Named alongside it

The objects these essays reach for when they reach for this one.

PriorCoverageCredible intervalInterval widthPrior sensitivitySample sizeAssuranceBeta distributionBinomial proportionEffect sizeExact enumerationFlat prior

All concepts