Normal Distribution
nothing is called 95% until it has been counted
Concept

Stratification — where it appears

Dividing the units into cells by their baseline characteristics and randomising separately inside each, so the cells' counts stay even. It balances the cells it was told about and says nothing about a covariate it was not, which is the same limitation every balancing rule has.

Named by 11 essays across 5 fields — each of them below, with the objects they name alongside it.

What a median split can see. A standard normal covariate with its median marked, and the mean of each category as a vertical rule: -0.7979, 0.7979. A rule that balances the categories is balancing those numbers and nothing else, so the part of the covariate it can act on is the variance between them — 0.6366 of the total, which at two categories is exactly 2/π because the two half-normal means are ±√(2/π). The rest, 0.3634, is variation inside the categories that the rule cannot see and does not touch: the assignment within a category is still a coin. Everything the next figure measures is a consequence of this one, and it is available before any unit has arrived.

A covariate with no levels

Every balancing rule on this site reads a level. Age and blood pressure have none, so somebody cuts them into categories — and a median split can see exactly 2/π of a normal covariate, whatever the rule does with the halves.

continuous · Assignment Four rules, three shapes, and no ordering that survives. The variance of the unadjusted treatment estimate under each rule, as a fraction of the variance a coin gives, over 450 trials of 200 units each. Against a covariate that enters linearly the rule that reads the number nearly halves it. Against a threshold at 1 it removes about a fifth. Against a quadratic every rule here is at or worse than a coin — they are all optimising a criterion that is one over the variance of an estimate in a model this outcome does not obey, and a constraint that helps nothing still costs something. Nothing in a trial says which column it is in.

Balanced on the wrong function

A rule that reads a covariate's numbers halves the variance of the treatment estimate, if the covariate enters the outcome as a straight line. If it enters as a threshold the rule is worth a fifth of that, and if it enters as a curve every rule here is worse than a coin.

shape · Blocking A threshold in the tail is a threshold nothing balances. The share of a coin's imbalance in an indicator 1{x > c} that survives a rule which balances the covariate itself. The smooth curve is 1 − ρ² with ρ = φ(c)/√(p(1−p)), a closed form with no trial in it; the points are counted over 500 trials of 200 units at each threshold. At the median the two agree that about a third survives — the removed share is exactly 2/π — and by two standard deviations 86.9% survives. The closed form is exact in the limit and optimistic by a few points at this many units, because the rule balances the sample's mean rather than the population's.

A threshold in the tail

How much of a threshold's imbalance a balanced covariate removes is a correlation, and the correlation is a closed form. At the median it is exactly 2/π — the same 2/π a median split throws away — and two standard deviations out it is an eighth.

shape · Blocking One odds ratio in every stratum, and a different one marginally. Five strata with baseline risks from 5% to 85%, a treatment allocated by a coin in each, and a conditional odds ratio of exactly 2.5 throughout. The marginal odds ratio is 1.789. Nothing is confounded; the odds ratio is simply not a weighted average of odds ratios.

The change that is not confounding

Five strata, a treatment allocated by a coin in every one, and an odds ratio of exactly 2.5 in all five. The odds ratio computed on the pooled table is 1.789. Nothing is confounded — an odds ratio is not a weighted average of odds ratios, and the risk difference, on the same table, is exactly its own stratum value.

reversal · Simpson A cut at a quantile, and a cut at a value. Two rules that read identically in a protocol. One splits each covariate at its median; the other splits it at 1 on the covariate's own scale — a dose, a temperature, a clinical threshold. At a correlation of 0.5 the first removes exactly nothing of the interaction between its own two splits, under every marginal here, because a median split is a function of the sign of the latent normal whatever the marginal is. The second removes what the bars show, and it does so on a normal covariate too: the threshold sits at 1.000 on the latent scale rather than at zero, so it is 59.4% odd and 40.6% even. The exact zero was never about the cut; it was about the cut being at the median.

The cut that is not a quantile

A protocol that says split the covariate at a threshold and one that says split it at the median read the same and are different rules. One has an exact guarantee under every marginal and the other has none under any.

skew · Criterion What stratifying on a mediator estimates, and what it does not. The direct effect stays at -0.109 throughout, because it is defined with the mediator held fixed. The total effect runs from -0.218 to 0.027 and crosses zero at 1.5. On 15% of the sweep the two have opposite signs, and both are correct answers.

Conditioning on what the treatment caused

When the grouping variable lies on the path from treatment to outcome, the stratified answer is the direct effect and the aggregate is the total effect. Both are correct. Over 15% of a sweep of the indirect path they have opposite signs, and no arithmetic on the table says which question was being asked.

reversal · Simpson What balancing several numbers at once costs each of them. The criterion generalises without a word changing — the covariate imbalance becomes a vector and the correction a quadratic form — so the question is what it is worth rather than whether it can be done. At n = 200 with 200 trials per point, a rule balancing one covariate leaves 12.7% of a coin's imbalance in it; balancing eight leaves 23.2% in each. The assignment has a fixed amount of freedom and every covariate added takes a share of it. The rule degrades rather than failing: at eight covariates it is still four times better balanced than a coin, and the eight are being held simultaneously rather than in turn.

Balancing more than one number

The criterion generalises to several covariates without a word changing, which makes the question what it is worth rather than whether it can be done. Each one added takes a share of the assignment's freedom, and the imbalance left in every one of them rises.

continuous · Blocking How often a randomised trial of eighty shows a Simpson reversal on some baseline variable, against how many were tabulated. The treatment helps every patient equally. On one nominated, strongly prognostic variable the reversal appears in 4.1% of trials. Tabulating 2, 5, 10, 20, 50, 100 variables of mixed prognostic value, some variable reverses in 4.1%, 6.5%, 10.8%, 16.1%, 26.1%, 34.1% of trials; stratifying the randomisation on the most prognostic one gives 0.2%, 2.9%, 7.7%, 12.2%, 23.4%, 31.9%.

Twenty variables nobody stratified on

A randomised trial of eighty patients in which the treatment helps every patient by the same six points shows Simpson's reversal on one nominated, strongly prognostic baseline variable in 4.07% of trials. Tabulate twenty baseline variables of mixed prognostic value and some variable reverses in 16.07% of trials; a hundred, in 34.13%. Stratifying the randomisation on the strongest variable removes its reversal and leaves 12.17% and 31.93%: the protection is exactly as wide as the list it was given.

reversal · Simpson Post-stratifying a simulated 5% rate on a companion correlated 0.9: the share of its variance removed, by the number of strata. The best any function of the companion can do is 48.1%; its own rate at the same cut, 38.2%. Strata spent across the band where the conditional probability changes reach 95% of the best at 8 strata; equal-probability strata need 32. At eight strata the two remove 46.8% and 29.9%.

A companion cut into many

A simulated rate can be estimated within bands of a companion whose distribution is exact, and the bands weighted back by their exact probabilities. One band boundary at the target's threshold is the companion's own rate; infinitely many are the best any function of the companion can do, which for a 5% rate at a correlation of 0.9 removes 48.1% of the variance. Eight bands placed where the target's conditional probability is changing remove 46.8%; eight at equal probabilities remove 29.9%, and need thirty-two to catch up. On the lognormal t test's size, thirty-two bands of the logarithms' t statistic were worth 2.99 times the draws and two hundred and fifty-six, 2.14.

splits · Routes What two exact companions are worth to a simulated rate: the lognormal t test's size post-stratified three ways, by the number of cells. To first order, bands of the logarithms' t statistic are worth at most 2.77 times the draws; a 32 × 32 grid in the pair 4.68; bands of a score fitted to the pair on a thousand samples 4.64 at sixteen bands and 4.69 at sixty-four. Counted over runs of 4,000 samples, the grid of 1,024 cells was worth 1.97, with 94 cells a run left empty.

A companion with two coordinates

The logarithms' t statistic, cut into bands, is worth at most 2.77 times the draws to a simulation of the lognormal t test's size. It is one function of two exactly known statistics, the logarithms' mean and spread, and the pair is worth 4.64 when a score fitted to it on a pilot of a thousand samples is cut into sixteen bands whose probabilities are integrated exactly. A grid of cells in the pair needs a thousand cells to reach the same and, counted on runs of four thousand samples, is worth 1.97. Band weights estimated from a pilot instead of computed are worth less than the pilot's draws would have been had they simply been counted — exactly nothing at a pilot the size of the run.

splits · Routes Approximate weights against the size of the run. What sixteen bands of the logarithms' t statistic are worth to a simulation of the lognormal t test's lower-tail size, in multiples of the draws, against the size of the run, with the band weights from Student's t exactly and from three approximations to it. Exact weights are worth 2.709 at every size. Fisher's one-term expansion is worth 2.708 at four thousand draws and 2.523 at a million, and breaks even with counting at 23.7 million draws. The variance-matched normal is worth 1.849 at four thousand and breaks even at 14,698. The standard normal is worth 0.762 at a thousand and breaks even at 669: a bias that does not shrink is multiplied by every draw added.

Weights from an approximate law

A companion whose law is only approximately known can still be cut into bands, with each band's weight computed from the approximation. The weight's error is a bias that does not shrink, and the run multiplies it: the stratified rate is worth p(1 − p)/(v + Nb²) times the draws. Reading Student's t on nineteen degrees of freedom as normal breaks even at 669 draws; the normal with the t's variance at about fifteen thousand; Fisher's one-term expansion at twenty-four million. What decides the bias is how the weight errors line up with the band rates, not how large they are — Fisher's is wrong by sixty per cent in one band and biases the rate by six hundredths of one.

splits · Routes

Named alongside it

The objects these essays reach for when they reach for this one.

RandomisationClosed formMonte CarloVariance reductionThresholdBias-varianceBinningContinuous covariateCovariate balanceAggregationAllocation ruleBlocking

All concepts