Normal Distribution
nothing is called 95% until it has been counted
Field

A promise about two arms

The fixed-width interval whose coverage is exact was built for one mean, and almost nothing anybody runs an experiment for is one mean. The construction survives with the harmonic effective size in place of the block size, and four things do not: a unit of that size costs four observations rather than one, the second variance puts Neyman's allocation in reach of a blinded rule, the conservation identity comes up one degree of freedom per block short, and there is an exact estimator under either of two conditions and none under both.
The construction survives a difference of two weighted means. Coverage of δ̂ ± t√(S_D²/H) on b − 1 degrees of freedom, over 900 runs at a requirement of 0.3, where δ̂ is the block differences weighted by h_b = (1/m_A + 1/m_B)⁻¹ and H is their total. The theorem the one-mean field rests on goes through with h_b in place of the block size, and the reason is that the weights a weighted least squares decomposition needs are the inverse variances — which is exactly what h_b is. The stopping rule reads only within-arm within-block contrasts, so it is a function of nothing the interval reports, whatever it does with the block sizes. Each bar is within 2.9% of the level it claims.

A width promised for a difference

The exact fixed-width interval was built for one mean. Two arms make the target 42.7 units of effective size and each unit costs four observations, so the same promise about a difference costs 169.4 rather than 42.7 — and the theorem survives untouched with the harmonic size in place of the block size.

 Three sets of weights, five designs, and no estimator that is exact everywhere. Coverage of the same interval under three weightings. h_b is the inverse variance when the arms share a variance or the allocation is constant; equal weights are right when every block has the same two counts; the estimated precision weights are right in the limit and exact nowhere, because the decomposition needs the weights to be the constants they are only estimating. In the corner — two variances, changing sizes, changing allocation — the two exact estimators are the ones that miss, at 98.45% and 95.65%, and the one with no theorem behind it is at 95.05%. That is the whole statement: there is an exact estimator under either condition, and none under both.

Which weights are the inverse variances

There is an exact estimator when the two arms share a variance and another when every block has the same two counts, and between them they cover every trial anybody designs on purpose. In the corner where neither holds, both cover 98.45% instead of 95%, and the only estimator at its level is the one with no theorem behind it.

 Two arms leave one degree of freedom per block unaccounted for. Each point is one run. The one-mean field's identity is (b − 1) + (N − b) = N − 1, and every schedule moves along that line rather than off it. Two arms give the rule N − 2b and the interval b − 1, which come to N − b − 1 — short of the N − 2 two arms leave by exactly one per block, since a block's arm counts absorb one degree of freedom each and only one of the two directions carries the difference. The hollow points add what the block sums are worth, b − 1 more, and land on the total. The missing degrees of freedom are not lost; they are in a place the interval has to be shown it may read.

The degrees of freedom in the sums

One arm partitions N − 1 exactly. Two arms give the rule N − 2b and the interval b − 1, which is short by one per block — and the missing ones are in the block sums, which are correlated with the differences at −0.79 and are usable anyway.

 Free until the sums stop seeing what the differences see. Coverage with and without the block sums pooled into the interval's variance estimate. With one effect and one level they are free. With an effect that varies between blocks they are still free, because a block's sum picks that variation up exactly as its difference does. With a level that varies they make the interval 37% wider and conservative. And where the effect falls as the level rises — a ceiling, and not an exotic thing to suppose — the sums carry none of the between-block variation while the differences carry all of it, the pooled estimate is short, and the interval that uses it covers 88.75% on a width 20% narrower than the honest one.

What a two-arm rule may not pool

A spread computed "within the block" without the arm label carries a share of the effect, so the trial runs 173 observations at a null and 282 at an effect of 1.5. The stopping rule is reading the thing it exists to measure, and the phrase that produced it is one word long.

 The sums, pooled as they are and made independent. Coverage on four designs at a requirement of 0.9, 20,000 runs each, about 3.0, 3.6, 3.4, 3.0 blocks. Refusing the sums: 94.91%, 95.06%, 94.96%, 94.86%. Pooling them as they are: 94.98%, 93.67%, 96.17%, 94.93% — exact at balance, short when the allocation is unbalanced and the same in every block, long when it alternates. Pooling the residualised sums, each block's sum less the part of it its own difference predicts: 94.98%, 95.16%, 95.06%, 94.70%, on every design. Each bar is ±0.31%.

The sums made independent

The block sums are a free second copy of a two-arm trial's variance only at balance. With ten and two in every block, pooling them covers 93.67% on four blocks; swap the arms each block and it covers 96.17%. Take from each sum the part its own difference predicts and the copy is independent at every allocation.

 Coverage as the effect moves with the level. Coverage on ten and two in every block, with the level between blocks moving at a spread of 0.5 and the effect moving with it at each slope, 4000 runs each. At -3: refusing the sums 94.4%, pooling them 88.8%, residualised 97.0%; At -2.5: refusing the sums 94.3%, pooling them 88.9%, residualised 95.6%; At -2: refusing the sums 94.9%, pooling them 90.0%, residualised 93.8%; At -1.5: refusing the sums 95.1%, pooling them 91.7%, residualised 92.7%; At -1: refusing the sums 95.1%, pooling them 94.8%, residualised 94.0%; At -0.5: refusing the sums 95.1%, pooling them 97.2%, residualised 97.7%; At 0: refusing the sums 95.0%, pooling them 98.8%, residualised 99.6%; At 0.5: refusing the sums 95.0%, pooling them 99.4%, residualised 99.9%; At 1: refusing the sums 95.0%, pooling them 99.4%, residualised 99.9%.

The sums under a moving level

A trial's block sums are a free second estimate of its variance only while they see what the differences see, and a level that moves between blocks with an effect moving against it breaks that: pooled raw at a ceiling they cover 87.6% to 90.0%. Residualising the sums on an unbalanced design does not make them safe. It moves the failure. On ten and two, 90.0% at the ceiling becomes 93.8%, the trough leaves for a slope of about −1.5, and over the rest of the range the same estimate over-covers, at 99.9% on an interval half as wide again as it needs to be.

All essays