Empirical Bayes — where it appears
Named by 17 essays across 5 fields — each of them below, with the objects they name alongside it.
What the plug-in forgets
The shrinkage weight needs a population spread, and the population spread has to be estimated from eight numbers. Empirical Bayes estimates it, substitutes it, and proceeds as though it were known — and the interval that comes out covers 79% rather than the 95% it claims.
A group from the population's own tail
Partial pooling halves the total squared error when a group's own standard error equals the spread between groups. Every group whose true effect sits more than 1.73 population widths from the centre — 8.33% of a perfectly normal population — does worse than it would have with its own mean, and its loss grows without bound. Among eight groups with the spread estimated, the most extreme is worse off in 61.6% of datasets. Capping the shift at one standard error keeps the total at 0.528 of the unpooled error and holds every group under twice it.
A prior on the spread
Integrating over the population spread means putting a prior on it, which sounds like the objection rather than the repair. The prior's effect is measurable, it is invisible where the groups are clearly different, and the reflex choice for a scale parameter turns out not to have a posterior at all.
Estimates that are too alike
Posterior means give each group its least-error estimate, and as a set they are too alike: with each group's standard error equal to the population's spread, they spread 0.707 as widely as the truth. Beyond two population widths lie 2.28% of the true effects, 7.86% of the groups' own means, and 0.234% of the posterior means — a tenth of the truth. Rescaling the estimates to the right spread counts the tail exactly and costs 17% more squared error; summing each group's posterior chance of being beyond the line counts it without changing any estimate.
The prior the data estimates
A hierarchical model needs a population spread, and it does not ask for one. It reads τ off the distance between the group means — biased six per cent low, exactly zero on 53% of datasets where the groups are identical — and the prior stops being a belief.
When the spread estimates to zero
The usual estimate of a population spread is a difference of two positive quantities, clamped at zero. On a third of eight-group datasets with a real spread in them the difference comes out negative, the estimate is exactly zero, and every group is pooled completely on data that said no such thing.
The interval that integrates
A credible interval for one group in a hierarchy has to average over every value the population spread might take. That averaging is what makes it cover — 95.2% against the plug-in's 78.8% — and it costs 31% more width, a heavier tail, and a mixture rather than a normal.
The small groups one centre protects
In a league table of a hundred groups where larger groups do better on average, pooling every group towards one centre overstates the small groups by half a population width — 0.501 — because it pulls them towards an average they are not part of. The top ten barely notices. The bottom ten, the list that flags poor performers, holds 3.19 of the true bottom ten when pooled to one centre and 4.28 when the groups are ranked by their own noisy means. Pooling towards a centre that rises with size restores 4.35 and puts the small groups back at the bottom where 80.6% of the true bottom ten sit.
The fewest groups that can borrow
At three groups the estimator that shrinks towards its own data's mean returns the group means untouched, on every dataset, because its constant is J − 3. At two it expands instead of shrinking. And the number of groups at which partial pooling starts to be worth doing is five, or two, or never — it depends on how far apart the groups are.
Shrinking the arm that was chosen
A trial that carries the best of eight arms forward reports it as better than it is — by +0.123 when the eight arms are alike — and its second stage alone is unbiased but the least accurate of the estimates. Pulling the chosen arm's first stage back towards the eight arms' mean before combining removes most of the bias when the arms are alike (+0.020) and gives the smallest error of any estimate, 0.133 against the published 0.182. When one arm really is ahead by half a standard deviation, the same shrinkage pulls the true winner back towards the losers and understates it by 0.103, with more error than the published estimate. The weight is estimated from eight numbers, and it is right when the selection was noise and wrong when the selection was right.
Where the borrowing goes
Pooling cuts the total squared error across eight groups by 56%. Two of the eight take 61% of that reduction, the four best-measured groups share 11% between them, and the largest group gets 1.5% of what the smallest does. The headline is a fact about the groups nobody was asking about.
A prior that lets one arm stand out
Shrinking the chosen arm of eight under a normal prior understates a genuine winner by 0.103. A t prior on three degrees of freedom with the same spread understates it by 0.101 — the tail is too far out to reach a lead of three and a half standard errors. A prior with a fifth of its weight on a vague component understates it by 0.073, costs 0.010 of bias when the arms are alike, and still has more error than the unshrunk estimate. With sixteen arms the same mixture recovers three fifths of the loss, because a bell of fifteen shows its outlier; with eight, the arms put more than half their posterior weight on 'one stands out' in 0.4% of trials where one does.
One population, or two
Group effects from two clusters rather than one bell, with the same total spread. The analysis recovers the same population spread, uses the same weight for every group, and reports nothing unusual — while 46% of its estimates land in a region holding 6.6% of the truths.
A second stage sized by the first
A drop-the-losers trial can size its second stage by what the first stage says about the arms' shape — more patients when the chosen arm's lead looks like luck, fewer when it looks real. The rule does what it was built to do: at sixteen arms with a genuine winner it spends 51.0 patients a group when the interim chose the wrong arm and 24.2 when it chose the right one. And it buys nothing. Against a fixed second stage of the same average size, every estimate of the chosen arm errs by the same to within a few thousandths; the published combination's bias rises from 0.014 to 0.022, and the second stage alone, still unbiased, errs by 0.319 against 0.281.
Patients spent before the choice
A trial that picks the best of eight arms and then confirms it splits its patients between choosing and estimating, and the familiar shape — sixty an arm before the choice, thirty after — spends ninety per cent of a six-hundred-patient budget on the choice. Measured against the effect a reader is told about, the best arm's, that split is nearly right when one arm is modestly ahead and twice as wrong as it needs to be when the arms are alike. A split of thirty-two an arm before the choice is never more than 21.3% worse than the best split for whichever world holds; sixty an arm can be 98.8% worse. The two targets an estimate could serve — the chosen arm's effect and the best arm's — pull the split in opposite directions, and only one of them is what gets published.
The shape the estimates keep
Partial pooling was suspected of destroying the evidence of its own misfit: compressing a two-cluster population of group effects towards one centre before any test could see the clusters. With groups of equal size it destroys nothing, because every estimate moves the same fraction of the way to the same place and the estimates keep the means' shape exactly — except when the fit pools completely, as it does on 29.9% of eight-group datasets at a spread of one. The evidence was never there: a shape check has 6.5% power at eight groups. With groups of unequal size the pooled estimates see the clusters better than the raw means do.
A correction built on a retest
Tweedie's correction for regression to the mean needs the error variance, and a study gets it from a few people measured twice. Twenty of them are enough: with normal errors the estimated correction then beats the correlation's rule on 73% to 77% of studies of a thousand readings from a heavy-tailed population, though its variance is uncertain by a third. What twenty cannot reveal is the error's shape. Laplace errors of the same variance make the correction overstate the lead a top group keeps by about 0.17, the plain rule then wins on most studies, and a study needs about two hundred retest pairs before a check on its own differences notices.
Named alongside it
The objects these essays reach for when they reach for this one.
ShrinkagePartial poolingHierarchical modelMean squared errorVariance componentsAdaptive designMethod of momentsPosteriorPosterior meanPriorSample sizeSelection bias