Normal Distribution
nothing is called 95% until it has been counted
Field

Balancing what has no levels

Every balancing rule in the two fields before this one reads a level. Age, blood pressure and a baseline score have none, and the first thing that happens to them is that somebody invents some — a choice with a cost available in closed form before any data exists: a median split can see exactly 2/π of a normal covariate, so a rule that balances its two halves perfectly still leaves three fifths of a coin's imbalance. A rule that reads the number instead does not beat that by a factor; it beats it by a rate.
What a median split can see. A standard normal covariate with its median marked, and the mean of each category as a vertical rule: -0.7979, 0.7979. A rule that balances the categories is balancing those numbers and nothing else, so the part of the covariate it can act on is the variance between them — 0.6366 of the total, which at two categories is exactly 2/π because the two half-normal means are ±√(2/π). The rest, 0.3634, is variation inside the categories that the rule cannot see and does not touch: the assignment within a category is still a coin. Everything the next figure measures is a consequence of this one, and it is available before any unit has arrived.

A covariate with no levels

Every balancing rule on this site reads a level. Age and blood pressure have none, so somebody cuts them into categories — and a median split can see exactly 2/π of a normal covariate, whatever the rule does with the halves.

 Two rates, not a factor. The standard deviation of the covariate imbalance under three rules, at five trial sizes, 260 trials each, on log axes. The upper line is a coin: its slope is -0.489, against a closed form of exactly −½. The middle line is minimisation on a median split; its slope is -0.519 — the same rate — because inside a category the assignment is still a coin, and what it buys is the constant, 0.654 of a coin's at n = 200. The lower line is the rule that reads x and maximises the information about the treatment effect: slope -0.987, nearly twice as steep. Its advantage is therefore not a number that can be quoted — it is 0.258 of a coin's at n = 50 and 0.065 at n = 800, and it keeps going.

The rule that reads the number

Stop categorising and let the rule read the covariate itself. What it should minimise is not an invented distance but the variance of the effect being estimated — and what comes back is not a better constant but a different rate.

 Three analyses of the same trials, none of them wrong about the data. 320 trials at n = 60 with no treatment effect at all, so every rejection counted is a false one, and a covariate that drives the outcome with coefficient 1. The unadjusted comparison is at 5.94% after a coin — its level — and at 0.00% after the rule that reads the covariate: the design removed the imbalance and the analysis is still pricing it. Adjusting for the covariate gives 4.06%, and the rule's own reference distribution — hold the outcomes, re-run the rule 199 times, count — gives 3.13% against the 4.5% that 199 draws can deliver. The last of the three has to be told the assignment rule and nothing else, which is the one thing the experimenter certainly knows.

What the balanced trial is worth

A rule that reads the covariate removes three quarters of the imbalance. An analysis that does not know it happened prices the imbalance anyway, rejects one true null in two hundred instead of one in twenty, and finds a real effect less often than a coin-tossed trial does.

 What balancing several numbers at once costs each of them. The criterion generalises without a word changing — the covariate imbalance becomes a vector and the correction a quadratic form — so the question is what it is worth rather than whether it can be done. At n = 200 with 200 trials per point, a rule balancing one covariate leaves 12.7% of a coin's imbalance in it; balancing eight leaves 23.2% in each. The assignment has a fixed amount of freedom and every covariate added takes a share of it. The rule degrades rather than failing: at eight covariates it is still four times better balanced than a coin, and the eight are being held simultaneously rather than in turn.

Balancing more than one number

The criterion generalises to several covariates without a word changing, which makes the question what it is worth rather than whether it can be done. Each one added takes a share of the assignment's freedom, and the imbalance left in every one of them rises.

 What balancing a recorded number does to the true one. The imbalance three rules leave in the true covariate X when what they read is a recording W of it, at six reliabilities and n = 200, 600 trials each, as a share of a coin's exact 2/√n. The lower curve is √(1 − λ), the closed-form floor for any rule that reads W: X = λW plus a residual of variance 1 − λ that W says nothing about, and a coin is all any rule can be on that residual. The rule that reads the number sits on it — 0.125 at 1, 0.345 at 0.9, 0.452 at 0.8, 0.612 at 0.64, 0.718 at 0.5, 0.834 at 0.3 — while balancing W itself to 0.125–0.133 of a coin's throughout. The upper curve is √(1 − 2λ/π), the same floor for minimisation on a median split of W, and minimisation sits on that one. At λ = 2/π the rule that reads the number is exactly as good as a median split of an error-free covariate: a cut is a measurement with reliability 0.637.

A number read with error

A rule that balances the recorded covariate balances the true one only as far as the recording reaches. At a reliability of 0.8 it leaves 45% of a coin's imbalance in the covariate that drives the outcome, back on the coin's 1/√n rate — and a median split turns out to be a recording with reliability 2/π.

 Four rules and what each leaves. The share of a coin's imbalance each rule leaves in the true value of each covariate, at 200 units and a reliability of 0.8. one reading of X: 21.4% in X and 100.1% in Z; two readings of X: 12.3% in X and 105.0% in Z; one reading each of X and Z: 21.7% in X and 21.5% in Z; two of X and one of Z: 13.4% in X and 22.0% in Z. The floors in closed form are 20.0% for one reading and 11.1% for two.

A second reading or a second covariate

A rule that balances a recording balances the truth only as far as the recording reaches, and a second reading of the covariate raises that reach. So does a first reading of something else. At a reliability of 0.8 the first reading of a second covariate does more once that covariate matters a third as much as the first, and the threshold is exact: the square root of (1 − λ)/(1 + λ), with no trial size in it. A protocol's list of measurements turns out to be a small knapsack, and it is rarely filled by reading anything twice.

All essays