Normal Distribution
nothing is called 95% until it has been counted
Concept

Standardised difference — where it appears

A difference in means divided by the spread of the thing being differenced, which under a random split of n units into halves is exactly 2/√n. That exact 2/√n is what every rerandomisation tolerance is stated in units of, so a tolerance means the same thing at every trial size.

Named by 8 essays across 3 fields — each of them below, with the objects they name alongside it.

What a median split can see. A standard normal covariate with its median marked, and the mean of each category as a vertical rule: -0.7979, 0.7979. A rule that balances the categories is balancing those numbers and nothing else, so the part of the covariate it can act on is the variance between them — 0.6366 of the total, which at two categories is exactly 2/π because the two half-normal means are ±√(2/π). The rest, 0.3634, is variation inside the categories that the rule cannot see and does not touch: the assignment within a category is still a coin. Everything the next figure measures is a consequence of this one, and it is available before any unit has arrived.

A covariate with no levels

Every balancing rule on this site reads a level. Age and blood pressure have none, so somebody cuts them into categories — and a median split can see exactly 2/π of a normal covariate, whatever the rule does with the halves.

continuous · Assignment What the rule blocks is not where it splits. How much of the separating direction each probe carries, over 100 designs of 14 units whose admissible set is enumerated and split into two pieces. The deferral this field answers proposed the constraint's active set — which exchanges the tolerance box actually blocks — as a better probe than the design's own leverage, on the ground that leverage is a heuristic and the active set is the quantity. Modelled from the design and the tolerance, it reads 0.4272 against leverage's 0.5395, at 4.43 paired standard errors the wrong way. Counted exactly over the enumerated set — at a cost no trial can pay — it reads 0.3854, worse again. Both beat a random direction at 0.2622, so they are probes; neither beats the two the earlier field already had.

What the rule blocks

A balancing rule breaks the admissible set into pieces by refusing exchanges. Which exchanges it refuses is computable from the design and the tolerance alone, before any assignment exists — and it makes a probe.

blocked · Randomisation A weight that balances, and one that unbalances. The standardised difference between the arms on each covariate, integrated over the population rather than counted in a sample. Unweighted, the arms differ by 0.8310 on the first covariate and 0.6015 on the second, which is what makes the raw difference of arm means 2.7102 against a true average effect of 1.0000. Weighting each unit by one over its own assignment probability removes both differences exactly — -2.78e-17 and -5.69e-19, which is machine precision and not a small number — because the weighted density of the treated arm is the population's own whatever the propensity is. Weighting by a score fitted without the second covariate balances the first to 0.0035 and pushes the second out to 0.7057, further apart than doing nothing.

A score that balances

Weighting each unit by one over its own assignment probability drives the standardised difference between the arms from 0.8310 to 2.8×10⁻¹⁷ — exactly, not nearly. A score fitted without the second covariate leaves that covariate at 0.7057, further apart than doing nothing at all.

weights · Weighting Two rates, not a factor. The standard deviation of the covariate imbalance under three rules, at five trial sizes, 260 trials each, on log axes. The upper line is a coin: its slope is -0.489, against a closed form of exactly −½. The middle line is minimisation on a median split; its slope is -0.519 — the same rate — because inside a category the assignment is still a coin, and what it buys is the constant, 0.654 of a coin's at n = 200. The lower line is the rule that reads x and maximises the information about the treatment effect: slope -0.987, nearly twice as steep. Its advantage is therefore not a number that can be quoted — it is 0.258 of a coin's at n = 50 and 0.065 at n = 800, and it keeps going.

The rule that reads the number

Stop categorising and let the rule read the covariate itself. What it should minimise is not an invented distance but the variance of the effect being estimated — and what comes back is not a better constant but a different rate.

continuous · Assignment What balancing a recorded number does to the true one. The imbalance three rules leave in the true covariate X when what they read is a recording W of it, at six reliabilities and n = 200, 600 trials each, as a share of a coin's exact 2/√n. The lower curve is √(1 − λ), the closed-form floor for any rule that reads W: X = λW plus a residual of variance 1 − λ that W says nothing about, and a coin is all any rule can be on that residual. The rule that reads the number sits on it — 0.125 at 1, 0.345 at 0.9, 0.452 at 0.8, 0.612 at 0.64, 0.718 at 0.5, 0.834 at 0.3 — while balancing W itself to 0.125–0.133 of a coin's throughout. The upper curve is √(1 − 2λ/π), the same floor for minimisation on a median split of W, and minimisation sits on that one. At λ = 2/π the rule that reads the number is exactly as good as a median split of an error-free covariate: a cut is a measurement with reliability 0.637.

A number read with error

A rule that balances the recorded covariate balances the true one only as far as the recording reaches. At a reliability of 0.8 it leaves 45% of a coin's imbalance in the covariate that drives the outcome, back on the coin's 1/√n rate — and a median split turns out to be a recording with reliability 2/π.

continuous · Assignment Weights that balance a sample by construction. What three sets of weights leave of the standardised difference between the arms on each covariate, as a root mean square over 1200 samples of 600 units. The true propensity leaves 0.1317 and 0.1186 — a sampling error, since it is right about the population and knows nothing of the draw. A likelihood fit leaves 0.0770 and 0.0657, having absorbed part of the draw's imbalance as a side effect of fitting the treatment. Weights fitted so that each arm's weighted means are the sample's leave 1.4e-14 and 1.2e-14, which is the arithmetic's floor rather than a small number: the largest gap between a weighted arm mean and the sample mean in any draw is 9.8e-14.

A weight fitted to balance

Weights fitted so that each arm's weighted covariate means equal the sample's leave a difference of 1.4×10⁻¹⁴ between the arms and give the estimate a third of the variance of weights fitted by likelihood — 0.011883, within a relative 5.8% of the bound no estimator can beat. In the world where the assignment carries a square nobody named, the same exact balance leaves the square further apart than no weighting at all, and where the outcome carries it too the estimate is wrong by 0.6973 with an interval that covers 1.5%.

weights · Weighting One weighting told the means and one told the second moments, in five worlds. The bias of the fit to balance over 600 samples of 600 units in each world, fitted to the covariates' means and fitted to their means, squares and product. Told the means it is off by -0.0004, -0.0020, 0.0103, 0.6973, 0.2698 in the worlds with no square, a square in the assignment, a square in the outcome, a square in both and a cube in both; told the second moments, by -0.0009, -0.0000, 0.0009, -0.0045, 0.3099. Its interval covers 94.0%, 94.7%, 95.0%, 1.5%, 51.0% and 93.7%, 89.8%, 94.0%, 91.0%, 48.3%.

The moments a balance is told

Weights fitted to balance the covariates' means were wrong by 0.6973 in the world where both the assignment and the outcome carry a square. Told the squares and the product as well, the same construction is off by −0.0045 there and its interval covers 91.0%. The failure moves up a moment rather than away: with a cube in both, the second-moment balance is off by 0.3099 and leaves the cube twice as far apart as no weighting. And where overlap is thin, 37.0% of samples have no such weights at all.

weights · Weighting The bias a tolerance on the second moments leaves where a square drives both the assignment and the outcome. 600 samples a point. Bias at δ = 0, 0.02, 0.05, 0.1, 0.2, 0.5: −0.0074, 0.0500, 0.1353, 0.2782, 0.5473, 0.6983; coverage 89.7%, 89.5%, 72.4%, 26.5%, 1.0%, 1.0%. The tolerance spent in full at the outcome's slope on the square would give 2·δ·√2: 0.0566, 0.1414, 0.2828, 0.5657 at the four smaller values; balancing the means alone gives 0.6973.

A balance allowed a tolerance

Weights that balance the means exactly and the second moments only to within a tenth of a standard deviation were supposed to keep the second-moment repair and recover the samples exact balance loses. Where a square drives both the assignment and the outcome they keep neither half of the repair: the bias is 0.2782 against −0.0074 with exact balance, and the interval covers 26.5%, because the weights spend the whole tolerance — every second-moment gap sits at the edge of its band — and the outcome converts a gap of a tenth into bias at its own slope, 2·δ·√2 = 0.283. At thin overlap the same tolerance does recover samples: 89.0% have weights at all, against 62.0% with exact balance.

weights · Weighting

Named alongside it

The objects these essays reach for when they reach for this one.

Covariate balanceClosed formCovariate imbalanceInverse-probability weightingModel misspecificationPropensity scoreContinuous covariateCovariate-adaptive randomisationDoubly robustExperimental designMinimisationMonte Carlo

All concepts