Where the two schools agree

The dispute between the two schools is usually staged as a disagreement about what probability means. That framing makes it look unresolvable, and it hides the more useful fact: in the case that comes up most often in teaching, the two produce the same interval to machine precision.

The case #

Estimate the mean of a normal distribution whose variance is known. Put a flat prior on the mean — every value equally likely, before any data.

The likelihood is normal in the mean, the prior is constant, so the posterior is the likelihood renormalised: normal, centred at the sample mean, with standard deviation σ/√n. The central 95% of that posterior runs from x̄ − 1.96σ/√n to x̄ + 1.96σ/√n.

The frequentist interval is x̄ ± 1.96σ/√n.

Not approximately. The same expression, arrived at from a different definition, and the site’s gate asserts the endpoints agree to fourteen decimal places — a tolerance chosen so that any future divergence is a real change rather than arithmetic drift.

What the agreement is resting on #

The case is chosen and it is worth saying which of its features is load-bearing.

A flat prior on a mean is improper — it does not integrate — and the posterior is proper anyway, because the likelihood is integrable in μ. That is a property of a location parameter with known scale and it does not carry.

Put a flat prior on a variance, or on a proportion’s log-odds, and the same move produces either an improper posterior or a proper one with a quite different shape from the frequentist interval.

So the agreement rests on an improper prior happening to work, and the case where it happens to work is the one the two schools are usually compared in.

What is still different #

The endpoints agree and the statements do not, and the difference is not decoration.

The Bayesian statement is about this interval: given the data and the prior, the probability that the mean lies in [2.31, 4.07] is 0.95. It is a claim about the parameter, conditional on what was observed.

The frequentist statement is about the procedure: intervals built this way contain the true mean 95% of the time across repeated samples. It says nothing about [2.31, 4.07] in particular, which either contains the mean or does not.

So a reader who wants a probability statement about the parameter is entitled to one here — but only by adopting the prior, and the prior is what licenses it. The frequentist interval does not become a probability statement by having the same endpoints.

This is the cleanest available illustration that the numbers and their interpretation are separable. Two analysts can compute identical output and be saying different things, and neither is confused.

Why it also explains the misreading #

The coincidence explains why the misreading described in what the 95% refers to is so persistent and so rarely punished.

In the case taught first, reading a confidence interval as a credible interval gives the right numbers. The endpoints are correct, any decision based on them is the decision the Bayesian analysis would reach, and nothing goes wrong. A student who acquires the wrong interpretation receives no corrective feedback, because in the setting where they acquired it the wrong interpretation is numerically harmless.

The habit then travels to settings where it is not harmless, and by then it is established.

That is a more satisfying account of the error’s durability than carelessness, and it suggests the remedy is to teach the divergent cases early rather than to repeat the definition more firmly.

What breaks the agreement #

Three things, in rough order of how often they bite.

A prior that is not flat. The moment the prior carries information the posterior is pulled toward it, and the credible interval shifts off the confidence interval by an amount proportional to the prior’s weight. This is the intended behaviour rather than a failure, and the pull is computable in advance.

A parameter that is not a location. The agreement above depends on the mean being a location parameter, with the likelihood shifting rigidly as the parameter changes. For a proportion, a variance, a rate or a correlation that is false, and the two approaches part company even under a flat prior. The proportion case is measured in the previous essay: at n = 20 and p = 0.05 a flat-prior credible interval covers 92.5% and Wald covers 63.9%, which is not a disagreement about interpretation.

A flat prior that is not flat. The subtlest of the three and the one that undoes the naive version of the whole idea. A prior that is flat in one parameterisation is not flat in another: a uniform prior on a proportion is not uniform on its log-odds, and the two give different posteriors. So “no information” does not pick out a unique prior, and the flat prior in the case above is doing real work rather than abstaining.

That third point is what Jeffreys’ prior exists to answer. It is constructed to be invariant under reparameterisation, so the answer does not depend on which coordinate happened to be written down — and its good coverage is a consequence of that construction rather than a target.

Matching priors, and how far the agreement extends #

The natural question after seeing one exact coincidence is how general it is, and the answer has a name.

A matching prior is one whose credible intervals have the correct frequentist coverage, either exactly or to a stated order in the sample size. For a location parameter the flat prior matches exactly, which is the case above. For most other problems no prior matches exactly, but priors exist that match to order 1/n — the coverage error shrinks faster than it otherwise would, so the agreement is asymptotic rather than exact.

Jeffreys’ prior is a matching prior in this sense for many one-parameter problems, which is the theoretical account of why its measured coverage in the previous essay is as good as it is.

Two things worth taking from the existence of that theory.

The agreement is not a coincidence of one example. There is a systematic relationship, the cases where the schools coincide are characterised, and the characterisation says the coincidence is common in simple problems and rare in complicated ones.

And it collapses in high dimensions. Matching priors are largely a one-parameter and low-dimensional phenomenon. With many parameters, a prior that is sensible for each one separately can be badly wrong jointly, and the coverage of the resulting intervals is not close to nominal. This is the setting where the choice between the schools has real consequences, and it is the setting neither is usually taught in.

The third route to agreement #

Worth stating because it is the one that resolves most real disputes.

With enough data, every prior that puts mass near the truth produces the same posterior, and that posterior is concentrated where the likelihood is. The credible interval converges on the confidence interval, and the question of which school to belong to stops having consequences.

The measurement from the first essay in this field puts a number on “enough”: four priors of very different weights, disagreeing sharply at ten observations, agree to within three points of the truth by eight hundred.

So the schools disagree in exactly the region where any two reasonable analyses disagree — small samples, weak information, decisions that hinge on how the uncertainty is described. Where the data is plentiful the argument is moot, and where it is scarce the disagreement between the schools is usually smaller than the disagreement between two frequentists about which interval to use, which is seven percentage points of coverage at n = 20.

What this suggests for practice #

Three things, none of which requires picking a side.

Know which case is at hand. For a mean with a flat prior the question is empty. For a proportion near the edge, a variance, or anything hierarchical, the choice changes the answer and should be made deliberately.

Report the interval and the interpretation separately. The endpoints are one thing and the claim attached to them is another, and the coincidence above proves they can be varied independently.

And where they diverge, say so and show both. A result that holds under a credible interval and a confidence interval is more robust than one that needs a particular reading, and computing both is cheap in every case this field covers.

Why the coincidence is worth teaching first #

A pedagogical claim, offered because the ordering in most courses is the reverse of this one.

The two schools are normally introduced as rival philosophies, with the technical differences following from the philosophical ones. That ordering makes the disagreement seem total, and it leaves a student unable to answer the only question that arises in practice: in this situation, does it matter?

Starting from the coincidence inverts the problem usefully. The default case is agreement; the question becomes what makes a case depart from it; and the answer is a short list — an informative prior, a non-location parameter, a high-dimensional model — that can be checked against the analysis in hand.

It also stops the philosophical disagreement from being mistaken for a numerical one. A student who has seen the same two numbers arrived at from both directions will not expect the choice of school to change an answer that it does not change, and will be alert when it does.

What the agreement does not license #

One caution, since the coincidence is the kind of fact that gets over-applied.

It is specific to the interval. Even where the endpoints agree exactly, the two frameworks continue to differ in what else they will do with the same model: what a hypothesis test means, whether it is legitimate to stop collecting data early, how a nuisance parameter is eliminated, and how several analyses are combined.

The stopping-rule question is the sharpest of those and it is the subject of a whole field on this site. A frequentist p-value depends on the sampling plan, so looking at the data repeatedly changes it; a Bayesian posterior does not depend on the stopping rule at all. Two analysts with identical data, identical models and identical interval endpoints can therefore disagree completely about whether an interim analysis was legitimate.

So the agreement is narrow and exact rather than broad and approximate, which is the useful shape for it to have. It settles one question completely and leaves the others untouched.

The summary worth carrying #

Three sentences.

For a location parameter with a flat prior, the credible interval and the confidence interval are the same interval, and the choice of school changes nothing but the sentence attached to the output.

Away from that case they diverge, by amounts this field measures rather than argues about — and at a proportion near the edge, the divergence is larger than most disagreements between two frequentists.

And with enough data both converge on the likelihood and on each other, so the dispute is confined to exactly the region where every other methodological choice also matters: small samples, weak information, and decisions that turn on how uncertainty is described.

The divergence that is easiest to miss #

One case deserves separating out, because it looks like the agreeing case and is not.

A mean with an unknown variance is the situation almost every real analysis is in, and the exact coincidence above assumed the variance known. With the variance estimated, the frequentist interval uses the t distribution — the correction that makes a small sample honest — and the Bayesian answer depends on what prior is placed on the variance.

Under one standard choice, a prior flat in the log of the standard deviation, the posterior for the mean is a t distribution with the same degrees of freedom, and the two intervals coincide again exactly. Under other perfectly defensible choices they do not.

So the agreement extends to the unknown-variance case, and it extends conditionally: it needs a particular prior on a parameter nobody was thinking about. The interval for the mean turns out to depend on an assumption about the variance, which is not obvious and is the kind of dependence that gets lost when a result is remembered as “they agree for a mean”.

The general shape of this is worth carrying. Coincidences between the two schools are usually exact rather than approximate, and usually conditional on something specific. Both halves matter: exactness means the agreement can be asserted to machine precision and checked, and conditionality means it has to be re-established rather than assumed whenever the setting changes.

What the field concludes #

The four essays here have taken the prior from a philosophical commitment to a component with a size, a cost and a measurable effect. The results, in the order they were established:

A prior is worth a stated number of observations, and for a Beta prior the number is a + b. That turns the central objection into a division.

A credible interval has a frequentist coverage, it can be summed exactly over the sample space, and on a standard non-informative prior it is better than the interval taught first — 95.7% against 87.6% at a sample of twenty.

A confident wrong prior produces intervals that never cover, which is the objection to priors stated as a measurement instead of a worry, and bounds exactly how bad the failure can be.

And in the case taught first the two schools give the same interval, which explains both why the distinction can be ignored for a long time and why the misreading of a confidence interval survives.

None of that resolves the disagreement about what probability is, and none of it needed to. The disagreement is real and it is mostly not what determines the answer; the prior’s weight, the shape of the parameter and the size of the sample are what determine the answer, and all three are numbers.

Where to look for the divergence in an analysis in hand #

The practical residue, as a checklist rather than a doctrine. The two schools will give materially different answers when any of these is true, and will not otherwise.

The prior is informative relative to the sample. Compute a + b against n and read off the answer.

The parameter is bounded or non-linear. A proportion near 0 or 1, a variance, a rate, a correlation near ±1. Anywhere the likelihood does not shift rigidly as the parameter moves.

The model is hierarchical or high-dimensional. This is where matching priors stop existing and where a prior that looks harmless on each parameter is not harmless jointly.

The stopping rule was not fixed in advance. Not a difference in the interval, but a difference in whether the frequentist calculation is valid at all — and the largest practical divergence between the two frameworks.

For anything not on that list, the choice can be made on grounds of what is convenient to compute and what sentence is wanted at the end, because the numbers will agree.

The list is short because the agreement is the default and the departures are specific. That is the opposite of the impression the philosophical framing leaves, and it is the more useful of the two pictures to carry into an actual analysis.

A last word on what “the same” means #

The agreement asserted here is numerical identity of two endpoints, checked to fourteen decimal places by the site’s gate. It is worth being precise about that tolerance, because a looser one would be claiming something weaker and easier.

Fourteen places is not a measurement of how close the two methods come. It is a statement that they are computing the same expression, and the only difference between the results is the order the floating-point operations happened to occur in. Had the agreement been merely close — agreeing to three or four places — that would be a different and much less interesting fact, and it would leave open the possibility of a small systematic difference hiding under the rounding.

Asserting it tightly is what makes the divergent cases meaningful. When a proportion near the edge produces a thirty-point gap in coverage, that gap is unambiguously a consequence of the setting rather than of arithmetic, because the same machinery agrees to machine precision when the setting permits.