Normal Distribution
nothing is called 95% until it has been counted
Concept

Flat prior — where it appears

A prior that spreads its weight evenly, offered as an expression of ignorance and never quite being one. It is not invariant to reparameterisation, so a flat prior on a variance and a flat prior on its logarithm are different assumptions about the same quantity.

Named by 8 essays across 3 fields — each of them below, with the objects they name alongside it.

weakly informative — Beta(2, 2), updated by 5 of 20. The prior is worth 4 observations. With 20 observations the posterior mean is 0.292, against a data proportion of 0.250 and a prior mean of 0.500.

What a prior is worth

A prior is not a philosophical position, it is a component with a stated size. For a proportion it is worth exactly a + b observations, which turns "how much does the prior matter" from an argument into a subtraction.

bayes · Prior What eight groups say about τ, when the truth is 1. The posterior density for the population spread after eight groups whose standard errors run from 0.5 to 2.1. The shaded band is the central 95% interval, from 1.02 to 4.44; the posterior median is 1.99 and the mean 2.18. The vertical mark at 1.07 is the moment estimate that empirical Bayes substitutes and then treats as known.

What the plug-in forgets

The shrinkage weight needs a population spread, and the population spread has to be estimated from eight numbers. Empirical Bayes estimates it, substitutes it, and proceeds as though it were known — and the interval that comes out covers 79% rather than the 95% it claims.

fullbayes · Shrinkage Three priors on the spread, at a true τ of 0.5. The posterior for τ under a flat prior (mean 1.66), a half-Cauchy of scale 1 (1.32) and one of scale 0.25 (1.15). The three answers differ by 30% of the widest. The prior does visible work when eight groups cannot separate a small spread from none, and almost none when they can.

A prior on the spread

Integrating over the population spread means putting a prior on it, which sounds like the objection rather than the repair. The prior's effect is measurable, it is invisible where the groups are clearly different, and the reflex choice for a scale parameter turns out not to have a posterior at all.

fullbayes · Prior What a 95% credible interval covers, n = 20. Computed by summing over all 21 possible counts rather than by simulating them. Jeffreys' prior covers close to 95% across the range; a confident prior centred in the wrong place covers almost nothing where the truth is far from it.

What a credible interval covers

A credible interval makes the statement everyone wants and does not claim to have a coverage. It has one anyway, it can be summed over the sample space exactly, and on a reasonable prior it beats the interval taught first.

bayes · Credible A normal mean with a flat prior — one interval, two readings. Both intervals are [1.878, 3.446]. The frequentist reading is that the procedure captures the truth 95% of the time; the Bayesian reading is that the parameter is in this interval with probability 0.95. The endpoints are identical to machine precision.

Where the two schools agree

With a flat prior on a normal mean, the credible interval and the confidence interval are the same interval, endpoint for endpoint. Knowing exactly when that stops being true is more useful than either camp's general argument.

bayes · Credible Counted coverage of two 95% intervals, over 2,000 datasets. Each point is one of the eight groups, at its own standard error. The integrated interval covers 95.2% overall against its stated 95%; the plug-in covers 78.8%, and its shortfall grows with the group's standard error — from 86.1% at se 0.5 to 77.0% at se 2.1. The mean widths are 3.48 and 2.66.

The interval that integrates

A credible interval for one group in a hierarchy has to average over every value the population spread might take. That averaging is what makes it cover — 95.2% against the plug-in's 78.8% — and it costs 31% more width, a heavier tail, and a mixture rather than a normal.

fullbayes · Credible The p-value of a study with 80% power, twenty thousand times. Twenty thousand two-sided z-tests, each on 25 observations whose true mean is 0.5603 standard deviations from the null, a noncentrality of 2.802. The bars are the counted share of p-values in bins a quarter of a power of ten wide, with the leftmost bin holding everything smaller; the line is the closed form. The middle eighty per cent of the p-values runs from 4.4×10⁻⁵ to 0.13, 3.46 orders of magnitude, the median is 0.0051, and 80.0% fall below 0.05, which is what the power means.

The p-value a replication gets

Under a true null a p-value is flat. Under a real effect its distribution is closed form and wide — a study with 80% power returns anything from 4.4×10⁻⁵ to 0.13 in eight runs of ten — and the chance that an exact replication of a p = 0.05 result is significant again is exactly one half, under both of the models people use without naming them.

testing · Uniformity A prior worth 35 observations, moved across the range — truth 0.1, n = 20. The same prior weight centred at each of 33 places. Its interval covers 100.0% where the centre is near the truth and 0.0% at its worst, while the mean width where it covers least is 0.221 against a flat prior's 0.263 on the same data.

When the prior is confident and wrong

A prior worth thirty-five observations, centred in the wrong place, produces a 95% interval that covers nothing at all — and reports a width 5% narrower than an honest one. It takes seventeen thousand observations to repair, not thirty-five, and the worst study to run is the one whose sample size equals the prior's weight, exactly.

bayes · Credible

Named alongside it

The objects these essays reach for when they reach for this one.

PosteriorPriorCoverageCredible intervalEmpirical BayesHierarchical modelPosterior meanPrior sensitivitySample sizeShrinkageMarginal likelihoodBeta distribution

All concepts