The slope that borrows

Everything in the pooling field so far has borrowed a mean. Eight hospitals’ average outcomes, eight schools’ average scores, eight batches’ average yields. The weight that decides how much each one borrows is

B = se² / (se² + τ²)

and in every picture of it, the groups are ordered by size and the arrows get shorter as the groups get bigger. It is very easy to come away believing that partial pooling is about sample size — that the rule is small groups borrow more.

That belief is wrong, and the cleanest way to show it wrong is to pool something that is not a mean.

What a slope’s standard error is made of #

For a mean of n observations with spread σ, the standard error is σ/√n. Sample size is the only thing in it, which is why pooling a mean looks like a statement about sample size.

For a least-squares slope it is

se(β̂) = σ / √Σ(x − x̄)²

and the sum in the denominator is not a count. It is a statement about the design: how far apart the x values were placed. Ten observations bunched into a narrow range of x contribute very little to it; ten spread wide contribute a great deal. The number of observations enters only through how many terms are in the sum, and it is dominated by how large each term is.

So two groups of ten can have slope standard errors that differ by a factor of five, and the shrinkage weight — which is a function of the standard error and nothing else — differs accordingly.

The five-point version, which is the one worth remembering:

Ten observations in every row.

The rule, restated #

The correct statement of what B measures has been available all along and is easy to skate past: how much a group borrows is decided by how well the group measures the quantity being pooled. Sample size is one input to that and often the dominant one; it is not the quantity.

This connects the pooling field to the design field in a way neither had made explicit. The variance removed before the data is about arranging a study so that the quantity of interest is measured precisely, and the arrangement is a choice made before anything is observed. What this essay adds is that the same choice decides, in advance, how much each group’s own data will be listened to in a pooled analysis.

A study that measures its groups over a narrow range of x has, without anyone deciding it, arranged for those groups to be told what to be by the population. A study that spreads its x values has arranged for them to speak for themselves. Neither is wrong; both are consequences of the design, and both are knowable before a single outcome is recorded, in exactly the way the imbalance under randomisation is knowable in advance.

The second route #

The claim that the pooled slope is a posterior mean is an identity, and this site does not accept identities as evidence that the implementation matches them.

Two routes, sharing only the least-squares fit itself. The first is the weighted average: compute the group’s own slope and its standard error, compute B, take B·(population slope) + (1 − B)·(own slope). The second writes down the group’s own likelihood as a function of the slope — the residual sum of squares with the intercept profiled out — multiplies it by the population’s normal density over slopes, and integrates on a grid of twenty thousand points. The second route never mentions a standard error, a weight, or a shrinkage factor.

They agree to 6 × 10⁻¹⁵, which is machine precision, at every one of six groups whose weights run from 0.89 to 0.25.

That agreement is worth more here than it was for a mean, because there was a real opportunity to get it wrong, and the wrong version would have looked entirely reasonable.

The standard error of a slope is not the standard error of a mean, and the habit of writing σ/√n is strong. Substituting it here — σ = 1.5, n = 10, so se = 0.474 — gives every one of the six groups the same standard error and therefore the same weight, B = 0.38 for all of them, against the true range of 0.89 down to 0.25. The picture that comes out of that is six parallel arrows of equal length. It is a perfectly plausible figure, it has no error in it anywhere that a build could detect, and it is a picture of the thing this essay says is not true.

Nothing in the first route would catch it, because the first route is where the mistake would be. The grid route has no standard error in it at all — it works from the residual sum of squares directly — so it cannot make that substitution, and it is the only thing on the page that would notice.

What it is worth #

Three estimators, run against two thousand datasets from a known truth, scored on total squared error in the slopes:

Partial pooling beats treating the groups as unrelated by a factor of 3.7 and beats treating them as identical by 1.6. That is a larger margin than the same comparison on means, and the reason is the spread of standard errors: when the groups are measured to very different precisions, an estimator that weights them by precision has more to gain than when they are all alike.

The middle row is the one to look at twice. Fitting one slope for all six groups — the answer a reader gets by ignoring the grouping and running a single regression on all sixty observations — is less than half as bad as fitting six separate slopes, on data where the six slopes genuinely differ. That is not an argument for ignoring the grouping; partial pooling beats it comfortably. It is a reminder of how bad six separate fits are when each is based on ten points, and it is the same arithmetic that makes more data is not monotone worth reading: the estimator that uses the most relevant data is not always the one that uses only the relevant data.

The standard error is one over the span #

Every number in the table lies on a single curve, and extracting it says what a design is actually buying.

Multiply each row’s span by its standard error. 0.4 × 1.86 = 0.744. 0.8 × 0.93 = 0.744. 1.2 × 0.62 = 0.744. 1.6 × 0.46 = 0.736. 2.0 × 0.37 = 0.740. The two readings taken off the design curve rather than out of the table land on the same constant — 0.2 × 3.72 = 0.744, and 2.9 × 0.25 = 0.725 — and so does the half-way point, since 0.744 divided by 1.25 is 0.595 against a population spread of 0.60. Seven readings, taken at different places for different purposes, one constant.

A slope’s standard error is inversely proportional to the span of its x values, and that is the whole content of the denominator once ten points are spread evenly across a range: Σ(x − x̄)² is n times the variance of those values, and the variance of evenly spread values is a fixed fraction of the square of their span. So se(β̂) goes as 1/(√n · span), and the two factors in that expression are not bought at the same price.

The consequence is arithmetic anybody designing a study can do before running it. Doubling the span halves the standard error; halving the standard error with observations instead takes four times as many. Read the ends of the table on those terms: the widest row’s standard error is 5.03 times smaller than the narrowest’s, and buying that factor by sampling would take 5.03² — a little over twenty-five — times the observations. Ten points spread over a span of 2.0 measure a slope as well as two hundred and fifty bunched into a span of 0.4.

That is the size of the asymmetry the two designs above were compared on. Sixty measurements laid out narrowly buy, for the purpose of estimating slopes, what fewer than three measurements per group would have bought laid out wide.

The design curve is a curve in one product #

Substituting se = c/span into the weight, with c the constant just measured, collapses the whole picture:

B = c² / (c² + τ²·span²)

The span and the population spread appear only as their product. So there is one design curve, not a family of them — τ stretches its horizontal axis and changes nothing else — and the half-way point, where a group borrows and keeps in equal measure, sits wherever τ·span = c. At τ = 0.6 that is a span of 0.744/0.6 = 1.24, which is the 1.25 the curve reports.

The same expression reproduces the near-identical population without redrawing anything. At τ = 0.15 a group spanning 2.9 has c/span = 0.257, and 0.257²/(0.257² + 0.15²) is 0.746 — the 74% that figure reports, arrived at from the constant rather than from the fit. The half-way point moves to 0.744/0.15 = 4.96, which is why it is nowhere in the drawn range.

This is worth having because it makes the design consequence quantitative in the only currency an experimenter controls. A group’s fate in a pooled analysis is decided by how its span compares with a single length, c/τ, and c is fixed by the noise while τ belongs to the population. A design can be judged against that length before any outcome exists, in exactly the way imbalance under randomisation can be. Spanning less than c/τ is a decision to be told what to be; spanning more is a decision to speak, and the cost of either was settled when the x values were chosen.

Two designs, the same budget #

The design consequence is worth working through as a decision rather than as an observation, because it is one somebody has to make.

A study has sixty measurements to spend across six groups. Two arrangements are available. Spread each group’s ten x values over a wide range, and each group measures its own slope to a standard error of 0.37; or bunch them into a narrow range, and each measures its slope to 1.86. The first design’s groups will borrow 28% from the population and the second’s will borrow 91%.

Which is better depends on a quantity nobody has yet: how much the groups actually differ. If they differ a great deal, the wide design is much better, because each group’s own slope is the only thing that knows about that group and the narrow design has thrown it away. If they hardly differ at all, the two designs give nearly the same answers, because the population is doing the work either way and the population’s slope is estimated from all sixty observations in both cases.

The asymmetry is what makes this a decision rather than a toss-up. The wide design is never much worse and is sometimes much better, so the wide design wins — and it wins for a reason that has nothing to do with pooling. Spreading x values is simply the way to measure a slope, and the variance removed before the data says the same thing about blocking.

What is new is that the arrangement determines the analysis as well as the precision. An analyst who receives the narrow design and applies partial pooling will produce six estimates that are nearly identical and six intervals that are nearly the same width, and will be tempted to report that the groups do not differ. They may or may not; the study cannot tell, and it could not have told before it started.

Two things this does not change #

It is worth being clear about what is the same as the one-parameter case, because the differences above are easy to over-read.

The weight is still the same weight. No new formula appears. B = se²/(se² + τ²) is applied to a quantity whose standard error happens to be computed differently, and everything derived about it in the weight that decides — that it is the posterior weight, that it is the squared-error minimiser, that James–Stein arrives at nearly the same estimator from a completely different argument — carries over unchanged.

The population spread is still estimated from as many numbers as there are groups. Six slopes give six numbers about how much slopes vary, and everything the previous field established about that applies here: the moment estimate has an atom at zero, the plug-in interval is too short, and integrating over τ is the repair. The figures in this essay use a known τ so that the design effect is not confounded with the estimation of τ, and an applied analysis would have neither.

Where this connects to leverage #

There is a fact in the regression field that is the same fact from the other side, and putting them together is the point of having both.

The line that one point drew establishes that an observation’s influence on a slope is decided by where its x sits, before anything is measured. This essay establishes that a group’s influence on its own pooled estimate is decided by where all its x values sit, before anything is measured. They are the same quantity — Σ(x − x̄)² — appearing once as leverage within a group and once as precision between groups.

The practical consequence of putting them together is uncomfortable. A group whose x values are spread wide measures its own slope well, so it borrows little from the population — and it is also the group whose slope is most vulnerable to a single extreme observation, because spreading x values is exactly what creates high-leverage points. The design that makes a group self-reliant is the design that makes it fragile.

Partial pooling does not know that. B is computed from the standard error, the standard error is computed from Σ(x − x̄)², and neither notices that one term is doing most of the work. Two groups with the same Σ(x − x̄)² — one with ten evenly spread points, one with nine bunched and one far away — receive identical weights and deserve very different ones.

That is a real limitation and it is not repaired anywhere in this field. What repairs it is the diagnostic machinery applied group by group before the pooling happens, which is a procedure rather than an estimator.

What the next essays take #

Three things have been established here and each is the seed of one of the essays after it.

The quantity being pooled has a standard error, and what that standard error is made of depends on the quantity. For a slope it is the design. The next essay takes a quantity where it is neither the sample size nor the design but the value being estimated: a proportion near zero carries far less information than one near a half, so how much a group borrows depends on what it observed.

The population the groups are drawn from has so far been a single number to shrink towards. It need not be. If something is known about each group before its outcome is measured, the groups can be shrunk towards a fitted line instead of a point, and the spread they borrow against becomes the scatter about that line rather than the scatter about the mean.

And the hierarchy has been two levels throughout. Adding a third does not change the arithmetic, and it produces a quantity — the design effect — that turns out to be the same quantity the time-series field computes for autocorrelated observations, arrived at from an entirely different picture.

What holds across all four is the sentence this essay exists to correct, and it is worth setting down in its final form. Partial pooling is not about how much data a group has. It is about how much a group’s data knows about the thing being estimated, and every essay in this field takes that same weight to a place where the answer is decided by something other than a count — by where the observations were placed, by what they turned out to be, by what was already known about the group, and by how many groups there are above it.