A budget of 4,000, at 1 and 20 a unit
Every affordable pair, enumerated. The best is 280 cheap units and 186 expensive ones — a ratio of 1.51, against the σᵢ/√cᵢ rule's 1.49. The unit rule, which says buy in the ratio of the spreads, lands at 66:197 and costs 17% more variance for the same money. Both rules are right about their own constraint; only one of them was asked.
Splitting the unitsslider: cost of a unit in the second arm, 5 positionswide8 views
What else it draws
The same object, drawn to answer the other questions the essays put to it.
6,000 experiments at each split with no difference between the arms at all, analysed two ways. The pooled t test rejects 0.00% of true nulls at 10:90 and 37.4% at 90:10, because it builds one estimate of σ from both arms and weights it by degrees of freedom while the standard error weights by 1/n. Welch's test, which keeps the two variances apart, holds 5.17% across the whole range. The variance-optimal split is marked, and it is in the conservative half.
Every control size, enumerated. The best is 132 on the control and 76 on each arm — a ratio of 1.74, against √3 = 1.73. Splitting the units evenly over all 4 groups costs 7.2%, which is small; what the larger control also does is lower the correlation between the comparisons, from 0.50 to 0.37, and that changes which multiplicity correction is right.
8,000 trials of 8 arms against one control with no effect anywhere, and four rules applied to each. Uncorrected, 24.5% of trials report a finding. Bonferroni holds 3.67% — below its claim, because it treats 8 comparisons that share a control as 8 unrelated ones — and Dunnett, which integrates over the control's error at ρ = 0.50, holds 4.50%. The difference is power on whatever arm is real.
Each point is one integer split, with its variance computed exactly rather than simulated. The minimum is at 25:75, which is the ratio of the spreads 25:75, and equal allocation costs 25% more variance — the same as throwing away 20 of the 100 units. The shaded band is every split within 5% of the best, and it runs from 17% to 35%: sharp to state, flat to sit on.
Each point is 6,000 two-stage experiments of 100 units: a pilot of m per arm, then the rest split by the pilot's own estimate of the two spreads. Above the line the pilot has made the experiment worse than not bothering. The best pilot here is 8 per arm at 0.809, against 0.800 for a designer who knew the spreads — so the rule recovers 96% of what knowing them is worth. A larger pilot estimates the ratio better and has less left to apply it to, which is why the curve turns.
The variance cost of an even split relative to the variance-minimising one for a risk difference, against the first arm's proportion, with the second at 0.3. The cost is a pure number: it does not depend on the trial's size. It is exactly zero at 0.3 and at 0.70, where the two arms have the same p(1 − p); it is 0.19% at a half and 4.36% at a tenth. Across the whole range from a tenth to nine tenths it never exceeds 4.36%, which is what the variance-minimising rule is worth here — and what it is worth is the reason it is safe to use with a guess.
The variance-minimising allocation for each of three ways of reporting the same two-arm comparison, against the first arm's proportion, with the second at 0.1. A risk difference wants the arm with the larger p(1 − p) to get more units; a log odds ratio wants it to get fewer, and the two curves are exact reflections of each other in the half line. A log risk ratio wants something else again. At a first-arm proportion of 0.6 they ask for 62.0%, 21.4% and 38.0% of the units. A trial reporting more than one of them cannot be optimal for either.
Where it is used
6 essays draw this figure, each at the numbers its own argument is about, so the same picture answers 6 different questions.
- Not half and half Splitting the units
- The cost of a unit Splitting the units
- One control, many arms Splitting the units
- Allocating on a guess Splitting the units
- The arm whose variance is its answer Splitting the units
- Two contrasts, one split Splitting the units