Four rules, and the floor two of them cannot pass
The standard deviation of the covariate imbalance under each rule at n = 200, over 500 trials, as a share of a coin's — which is exactly 2/√n = 0.1414 and is the one number here that needs no simulation. Blocking inside 2 categories and minimising on the same 2 categories are the same rule to within their noise, 61.7% and 61.2%, and the marked line is why: a 2-category split can see 63.7% of the covariate's variance, so a rule that balanced its categories perfectly would still leave 60.3% of a coin's imbalance. Reading the number instead leaves 13.0%, and the worst imbalance it produced in 500 trials was 0.094 standard deviations against the coin's 0.439.
Balancing what has no levelsslider: how many units are in the trial, 4 positionswide3 views
What else it draws
The same object, drawn to answer the other questions the essays put to it.
A standard normal covariate with its median marked, and the mean of each category as a vertical rule: -0.7979, 0.7979. A rule that balances the categories is balancing those numbers and nothing else, so the part of the covariate it can act on is the variance between them — 0.6366 of the total, which at two categories is exactly 2/π because the two half-normal means are ±√(2/π). The rest, 0.3634, is variation inside the categories that the rule cannot see and does not touch: the assignment within a category is still a coin. Everything the next figure measures is a consequence of this one, and it is available before any unit has arrived.
320 trials at n = 60 with no treatment effect at all, so every rejection counted is a false one, and a covariate that drives the outcome with coefficient 1. The unadjusted comparison is at 5.94% after a coin — its level — and at 0.00% after the rule that reads the covariate: the design removed the imbalance and the analysis is still pricing it. Adjusting for the covariate gives 4.06%, and the rule's own reference distribution — hold the outcomes, re-run the rule 199 times, count — gives 3.13% against the 4.5% that 199 draws can deliver. The last of the three has to be told the assignment rule and nothing else, which is the one thing the experimenter certainly knows.
Where it is used
8 essays draw this figure, each at the numbers its own argument is about, so the same picture answers 8 different questions.
- A covariate with no levels Balancing what has no levels
- Balanced on the wrong function The shape the covariate enters by
- A threshold in the tail The shape the covariate enters by
- The rule that reads the number Balancing what has no levels
- Three functions of one number The shape the covariate enters by
- What the balanced trial is worth Balancing what has no levels
- Balancing more than one number Balancing what has no levels
- The analysis and the shape The shape the covariate enters by