The sums made independent

A two-arm trial run in blocks, with a stopping rule that reads only within-arm contrasts, gives an exact interval for the difference between arms built from the block differences. The essay before this one found a second copy of the same variance lying unused: each block’s sum of arm means has exactly the variance its difference has, so the sums can be pooled into the interval’s spread estimate and double its degrees of freedom. At balance that is free — a narrower interval, the same coverage.

Off balance the sum and the difference are correlated, and the earlier essay showed why that does not matter as long as the allocation is the same in every block: the reported difference is linear in the block differences, the sums’ spread is a quadratic about their own weighted mean, and with a constant correlation the coupling term cancels. It named the case where it does not cancel — an allocation that changes between blocks, where the parity argument’s cancellation fails and which is not separately measured — and stopped there.

This essay measures that case, and on the way finds that the constant-allocation case was not quite free either. Both problems have the same repair.

Four designs, three ways to use the sums #

Every measurement here uses blocks of twelve observations, a blinded rule stopping when the accumulated precision reaches its target, arms with a common variance, and twenty thousand runs per row. Four allocations: six and six in every block; ten and two in every block; ten and two, then two and ten, alternating; and eight and four, then four and eight. And three intervals on each run: the honest one that refuses the sums, the one that pools them as they are, and a third described below.

At a requirement loose enough that trials stop after about three or four blocks, refusing the sums covers 94.91%, 95.06%, 94.96% and 94.86% on the four designs, as the construction guarantees. Pooling them as they are covers 94.98% at balance — free, as before — but 93.67% with ten and two in every block, and 96.17% when ten and two alternates with two and ten. Each is twenty thousand runs with a standard error of 0.15 points: the first failure is more than eight standard errors short and the second more than seven over.

The constant allocation: a copy that is not independent of the original #

The ten-and-two design breaks no parity argument. The cancellation holds, and the sums’ spread estimate is uncorrelated with the squared error of the reported difference — 0.0064 at this requirement, nothing. What fails is the counting.

Within a block the sum and the difference of two arm means are correlated by

$$\rho_b \;=\; \frac{1/m_A - 1/m_B}{1/m_A + 1/m_B},$$

which is zero at balance and −0.667 at ten and two. The two estimates of $\sigma^2$ they produce, one from the differences and one from the sums, are then correlated with each other: measured over the runs, 0.435, against $\rho^2$ = 0.444. Pooling two correlated estimates does not give twice the degrees of freedom of one. The interval that pools them claims 2(b − 1) and has fewer, so its t quantile is too small and it under-covers.

The obvious fix is to count them properly. Two estimates with correlation $\rho^2$ between them are worth $2(b - 1)/(1 + \rho^2)$ degrees of freedom in variance, and a t interval on that count covers 95.84% on the ten-and-two design — now over. The count is right about the variance of the pooled estimate and wrong about its shape: a sum of two correlated $\chi^2$ variables is not a scaled $\chi^2$ on any number of degrees of freedom, so no t quantile makes the interval exact. Recounting moves the error across the line rather than removing it.

The alternating allocation: the parity argument fails #

Swap the arms each block and $\rho_b$ alternates between −0.667 and +0.667. Now the cancellation the earlier essay relied on is gone: the covariance between a block’s centred sum and the reported difference is ($h_b$/H)$c_b$ minus a weighted average of the $c_j$, and with $c_j$ alternating in sign the average is near zero while each $c_b$ is not.

The consequence is visible directly. Over twenty thousand runs the correlation between the squared error of the reported difference and the sums’ variance estimate is 0.2796 on about three blocks, 0.1877 on about seven and 0.1304 on about twelve. With the allocation fixed it is 0.0064, 0.0030 and −0.0018 at the same requirements. Under the alternating design the sums’ estimate is large exactly when the error is large, so the pooled interval is wide when it most needs to be — and over-covers, at 96.17%, 95.62% and 95.38%. It also fails the other way at the same time, through the same correlated-copy counting as the fixed design, which is why recounting the degrees of freedom makes it worse: 97.89% on the loosest requirement.

An over-covering interval is not a safe one, for the reason these essays keep returning to. Its level now depends on the design, and an analyst who pools the sums on one trial with a run-in that favoured one arm and on another without has two intervals that mean different things.

The repair: take out what the difference predicts #

Both failures have one source. Each block’s sum carries a piece of its own difference — $\rho_b d_b$ in the best linear prediction — and that piece is what correlates the two copies in the fixed design and what couples the sums to the error in the alternating one. For jointly normal observations with a common variance, removing it leaves something independent of the difference entirely:

$$\tilde s_b \;=\; s_b - \rho_b\,d_b, \qquad \operatorname{Var}(\tilde s_b) \;=\; \sigma^2(1 - \rho_b^2)/h_b,$$

and $\tilde s_b$ is independent of $d_b$, and so of every block difference and of the reported estimate. Its mean is 2μ + δ(1 − $\rho_b$), which is constant when the allocation is, so $\tilde s_b$ is centred at its weighted mean; when the allocation alternates the mean moves with $\rho_b$, and $\tilde s_b$ is regressed on (1, $\rho_b$) instead, at the cost of one more degree of freedom. Either way its weighted residual sum of squares, with weights $h_b$/(1 − $\rho_b^2$), is $\sigma^2$ times a $\chi^2$ independent of everything the honest interval is built from. Adding it to the differences’ sum of squares gives a pooled estimate that is exactly a scaled $\chi^2$ on (b − 1) + (b − 1) degrees of freedom, or (b − 1) + (b − 2), and the t interval on it is exact.

For a block of ten and two the arithmetic is short. The effective size is h = (1/10 + 1/2)⁻¹ = 5/3, the correlation is ρ = (1/10 − 1/2)/(1/10 + 1/2) = −2/3, and the residualised sum is the block’s sum plus two thirds of its difference. Its variance is $\sigma^2(1 - 4/9)/h = \sigma^2 \cdot 5/9 \cdot 3/5 = \sigma^2/3$, a third of a single observation’s, and it enters the pooled sum of squares with weight $h/(1 - \rho^2) = 3$. Every quantity in that line is fixed by the two arm counts before the block is observed. Nothing in it needs the data, and nothing in it changes if the arms are swapped except the sign of ρ, which is exactly the sign the alternating design flips and the regression on (1, $\rho_b$) absorbs.

The measurements agree with the algebra. On the loose requirement the residualised interval covers 94.98%, 95.16%, 95.06% and 94.70% on the four designs; at a requirement where trials run to four to seven blocks, 95.06%, 95.12%, 95.06% and 94.97%; at one where they run to fifteen to twenty-six blocks, 95.05%, 95.08%, 94.84% and 94.97%. Every value is within two standard errors of 95%. The correlation between its variance estimate and the squared error, on the alternating design, is −0.0115, −0.0119 and −0.0050 at the three requirements of the mechanism figure: the coupling is gone.

At balance the residualised sums are the raw sums, because $\rho_b$ is zero, and the two pooled intervals are one.

Both failures belong to short trials #

The two failures of the raw sums have different mechanisms, and they fade at different rates as the trial runs to more blocks.

With ten and two fixed, the raw pooled interval covers 93.67% on about four blocks, 94.12% on six, 94.17% on ten and 94.70% on twenty-five. The shortfall is a matter of degrees of freedom — the interval claims 2(b − 1) and has fewer — and a t quantile on a few degrees of freedom is very sensitive to the count while one on fifty barely moves, so the shortfall dies away roughly as the difference between two t quantiles does.

With ten and two alternating, the raw pooled interval covers 96.17% on about three blocks, 95.62% on seven, 95.38% on twelve and 94.97% on twenty-six. Here the failure is the coupling to the error: the part of the sums’ spread that tracks the squared error is one term among 2(b − 1), so its influence on the pooled estimate falls as the blocks multiply, and the correlation in the mechanism figure falls with it, from 0.2796 to 0.1304 over the first three requirements.

Neither failure is large on a long trial, which is why a measurement at twenty-odd blocks — where the earlier essay made its — would see neither. But the sums are worth most exactly where both failures are largest: on a short trial, the honest interval’s b − 1 degrees of freedom are scarce, and doubling them narrows the interval by a quarter or more. A repair that holds only on long trials would hold only where it is not needed.

Where this sits in the blinded construction #

The whole field rests on a separation the one-mean construction made first: the stopping rule reads quantities independent of what the interval reports, and the interval is exact because of it. The essay that let the block size change showed the separation survives any schedule, and the one-mean identity showed where every degree of freedom goes. Two arms add the sums, which estimate the same variance and are not read by the rule at all — so they never threatened the separation between rule and interval. What they threatened is a second separation inside the interval itself, between the spread estimate and the estimate it divides.

That is the same independence the forbidden-spread dial measured from the stopping side, arriving here from the estimating side. A spread estimate coupled to the error it is meant to measure gives an interval whose coverage depends on the coupling, whichever part of the procedure introduced it. The residualised sums are the estimating side’s version of a rule that cannot see the mean: a quantity built so that, by construction, it carries nothing about the difference.

What the independent copy is still worth #

A repair that cost all of the gain would not be worth making. The residualised sums have smaller variance than the raw ones, but they are weighted up to match, so what they lose is only the information that was really in the difference already.

On the loose requirement, refusing the sums gives mean half-widths of 1.2675, 1.4365, 1.4919 and 1.3333 on the four designs. Pooling them as they are gives 0.8691, 1.0192, 1.0589 and 0.9155 — narrower, and wrong on two of the four. Pooling them residualised gives 0.8691, 1.0388, 1.1760 and 1.0302: on the ten-and-two design it keeps 95% of the narrowing, and on the alternating ones, where it fits one more coefficient, 73% and 73%. Against the honest interval it is still 28% narrower on ten and two fixed and 21% on ten and two alternating.

At a requirement where trials run to a couple of dozen blocks the three widths nearly coincide — 0.3322, 0.3227 and 0.3237 on the fixed ten and two — because with that many blocks the interval’s own degrees of freedom are no longer scarce and doubling them buys little. The sums are worth having on short trials, which are also the trials where pooling them naively fails.

What a protocol should take from this #

Pooling the sums is a decision about the allocation as well as about the variance. At balance it is free. With any unbalanced allocation it needs the residualised form, whether or not the allocation changes between blocks — the fixed case fails more quietly, in the under-covering direction, and is the one a reader of the earlier essay would have expected to be safe.

The repair needs only things the rule already has. $\rho_b$ is a function of the block’s two arm counts, which the design fixed. Nothing about it reads a mean, so a blinded rule may compute it, and the interval stays exact under the same conditions as the honest one: arms sharing a variance. With unequal arm variances $\rho_b$ depends on the variance ratio as well, the field’s corner where neither exact estimator survives, and the repair would need the ratio too.

Recounting is not repairing. The first correction anyone would reach for — fewer degrees of freedom for a correlated copy — gets the variance of the pooled estimate right and its distribution wrong, and over-covers by nearly a point on four blocks. The residualised sums are not a correction to the count; they change the estimate into one the count is exact for.

Still open: the sums under a moving level #

Every design here holds the level fixed across blocks. The earlier essay found that a level varying between blocks inflates the sums and not the differences, so naive pooling over-covers, and that an effect which falls as the level rises makes it under-cover by six points. The residualised sum carries the level with weight one and the effect with weight 1 − $\rho_b$, so under an unbalanced allocation a ceiling effect is partly carried into it and partly removed. Whether residualising softens the six-point failure on an unbalanced design, worsens it, or leaves it alone is the measurement that would say whether the sums can ever be pooled safely in a trial whose level drifts — which is most trials that run long enough to be blocked.

What is claimed here, and what is not #

This essay takes pooling the block sums into a blinded two-arm interval when the allocation is unbalanced. The claims are that on blocks of twelve with a common arm variance, naive pooling under-covers when ten and two is fixed — 93.67% on about four blocks — because the sums and differences give correlated copies of $\sigma^2$, correlated at $\rho^2$ = 0.444; that it over-covers when ten and two alternates with two and ten — 96.17% — because the parity argument’s cancellation fails and the sums’ estimate is coupled to the squared error, at a correlation of 0.28; that recounting the degrees of freedom does not repair either; and that the residualised sums $s_b$ − $\rho_b d_b$ give an exact pooled interval on every design measured while keeping most of the narrowing.

What stays out, named as a decision: unequal arm variances, where $\rho_b$ needs the variance ratio and the field’s corner applies; a drifting level, deferred above; and allocations that vary in other ways than alternating — a run-in of a few blocks at one ratio followed by a long stretch at another, say — which the regression on (1, $\rho_b$) handles by construction and which is not separately measured.

The boundary against the first essay of this field is the second copy: that essay owns the construction and the effective size, and the essay on what a two-arm rule may not pool owns the quantities a rule may not read. This one owns what the sums are worth once the allocation stops being balanced.

The checks, and the refusal that makes them mean something #

The coverage figure checks at every requirement that the residualised interval is within four standard errors of 95% on all four designs; the width figure that it is narrower than the honest interval on all four; the mechanism figure that the alternating design couples the raw sums to the error while the fixed design and the residualised sums do not. The refusal is the naive pooling on the alternating design claimed to be at its level on a loose requirement, which must fail — a check that passed it would be accepting an interval whose coverage depends on whether the allocation happened to swap.