Prior — the series · Normal Distribution

weakly informative — Beta(2, 2), updated by 5 of 20. The prior is worth 4 observations. With 20 observations the posterior mean is 0.292, against a data proportion of 0.250 and a prior mean of 0.500.

What a prior is worth

A prior is not a philosophical position, it is a component with a stated size. For a proportion it is worth exactly a + b observations, which turns "how much does the prior matter" from an argument into a subtraction.

part 1 · bayes

τ̂ across 2,000 datasets of 12 groups, true τ = 1.5. The population spread is not supplied to a hierarchical model — it is estimated from how far apart the group means are, after subtracting the noise that would separate them anyway. It averages 1.41 here against a true 1.5, and comes out exactly zero on 5% of datasets.

The prior the data estimates

A hierarchical model needs a population spread, and it does not ask for one. It reads τ off the distance between the group means — biased six per cent low, exactly zero on 53% of datasets where the groups are identical — and the prior stops being a belief.

part 2 · hierarchical

Three priors on the spread, at a true τ of 0.5. The posterior for τ under a flat prior (mean 1.66), a half-Cauchy of scale 1 (1.32) and one of scale 0.25 (1.15). The three answers differ by 30% of the widest. The prior does visible work when eight groups cannot separate a small spread from none, and almost none when they can.

A prior on the spread

Integrating over the population spread means putting a prior on it, which sounds like the objection rather than the repair. The prior's effect is measurable, it is invisible where the groups are clearly different, and the reflex choice for a scale parameter turns out not to have a posterior at all.

part 3 · fullbayes

How often a trial that borrows its control from an earlier one declares a treatment with no effect a success, by the earlier control's drift. Fifty patients an arm, an earlier control of two hundred, a one-sided 2.5% threshold. Without borrowing the rate is 2.5% at every drift. Pooling outright reaches 73.1% at a drift of half a standard deviation. A half-normal prior of scale 0.05 on the between-trial spread peaks at 35.2% and is still at 31.8% at a drift of 1.5; a half-Cauchy of the same scale peaks at 11.0% near a drift of 0.4 and falls back to 3.4%.

A control borrowed from the last trial

A trial of fifty patients an arm can borrow its control group from an earlier trial of two hundred by treating the two control means as draws from one population with a spread between trials. With two trials there is one difference to learn that spread from, so the prior on it decides how much is borrowed — and its tail decides whether the borrowing stops when the two trials disagree. A half-normal prior of scale 0.05 buys 17.4 points of power and, at a drift of 1.5 standard deviations, still declares a treatment with no effect a success 31.8% of the time. A half-Cauchy of the same scale buys 13.6 points, peaks at 11.0% and falls back to 3.4%.

part 4 · fullbayes

How often a trial borrowing from 16 earlier trials that agree declares a treatment with no effect a success, by its own control's drift. Fifty patients an arm, each earlier control of two hundred, a one-sided 2.5% threshold. With 16 earlier trials, the half-normal prior peaks at 88.1%, the half-Cauchy at 80.7% near a drift of 0.6, and the half-Cauchy with a fifth of the prior on a vague component at 16.9%. At a drift of 1.5 the three read 73.3%, 25.5% and 3.7%.

A history that agrees with itself

Borrowing a control from sixteen earlier trials that agree with each other should be safer than borrowing from one, and it is the opposite. Sixteen agreeing trials estimate the spread between trials as small, and a small spread estimated confidently is a licence to pool: under a half-Cauchy prior that let go of a single disagreeing trial, a current control 0.6 standard deviations from a sixteen-trial history turns a treatment with no effect into a success 80.7% of the time, against 10.5% at worst with one earlier trial. Moving a fifth of the prior onto a vague component brings the worst case to 16.9% and keeps power at 91.0%, against 70.5% without borrowing.

part 5 · fullbayes

How often a trial borrowing from 8 earlier trials on a trend declares a treatment with no effect a success. Fifty patients an arm, each earlier control of two hundred, the current trial one step after the last. At slopes 0, 0.01, 0.02, 0.03, 0.05, 0.075, 0.1: exchangeable 2.7%, 4.4%, 6.6%, 8.3%, 9.9%, 8.3%, 6.7%; with a fifth of the prior vague 2.4%, 3.6%, 5.5%, 6.9%, 7.6%, 6.5%, 5.7%; with a line in time 2.9% at every slope.

A history that has been moving

Earlier control arms borrowed as exchangeable draws are pooled towards the history's middle, and when the control mean has been rising the current trial sits at its end. A drift of 0.02 standard deviations a trial — a step between neighbouring trials of under a third of either one's standard error — turns a treatment with no effect into a success 9.9% of the time from sixteen earlier trials, and a drift of 0.05 overstates the effect by 0.130 standard deviations from eight. A line in time removes the bias exactly, at every slope, and pays for it in borrowing: from eight earlier trials it borrows 175 control patients where the exchangeable model borrows 439.

part 6 · fullbayes

The bias each model puts into the treatment effect when sixteen earlier controls stepped once. Sixteen earlier trials of two hundred, the first eight a step below the last eight, which sit at the current control's level; the current trial has fifty patients an arm and no treatment effect; 1,500 trials at each step. Exchangeable trials: -0.003, 0.020, 0.041, 0.056, 0.063, 0.065, 0.054. A line in time: -0.003, -0.016, -0.029, -0.041, -0.052, -0.067, -0.075. A change point: -0.003, 0.003, 0.003, -0.001, -0.002, -0.003, -0.003 — at steps of 0, 0.05, 0.1, 0.15, 0.2, 0.3, 0.5.

A history that stepped

When sixteen earlier control arms stepped once — the first eight a fifth of a standard deviation below the last — borrowing them as exchangeable flatters a treatment with no effect by 0.063 and stops borrowing, and fitting a line through them penalises it by 0.052. A model with a change point is unbiased at every step size, borrows more the larger the step, 512 patients at a fifth, and puts two thirds of its belief on the right place for the change. Against a steady drift it is the one that fails, so each of the three is safe only against the shape of history it assumes.

part 7 · fullbayes