Normal Distribution
nothing is called 95% until it has been counted
Concept

Baseline imbalance — where it appears

A difference between the arms of a trial in a variable measured before treatment. Randomisation makes it a matter of chance rather than of choice, so it is a source of variance in the estimated effect rather than of bias, and adjustment can remove that variance.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

How often a randomised trial of eighty shows a Simpson reversal on some baseline variable, against how many were tabulated. The treatment helps every patient equally. On one nominated, strongly prognostic variable the reversal appears in 4.1% of trials. Tabulating 2, 5, 10, 20, 50, 100 variables of mixed prognostic value, some variable reverses in 4.1%, 6.5%, 10.8%, 16.1%, 26.1%, 34.1% of trials; stratifying the randomisation on the most prognostic one gives 0.2%, 2.9%, 7.7%, 12.2%, 23.4%, 31.9%.

Twenty variables nobody stratified on

A randomised trial of eighty patients in which the treatment helps every patient by the same six points shows Simpson's reversal on one nominated, strongly prognostic baseline variable in 4.07% of trials. Tabulate twenty baseline variables of mixed prognostic value and some variable reverses in 16.07% of trials; a hundred, in 34.13%. Stratifying the randomisation on the strongest variable removes its reversal and leaves 12.17% and 31.93%: the protection is exactly as wide as the list it was given.

reversal · Simpson How often a trial with nothing to find shows a Simpson reversal somewhere, by how many variables are searched and how. Trials of eighty whose treatment helps every patient by six points, twenty baseline variables tracking the risk with correlations from 0.95 to 0.10; 4,000 trials. Searching the strongest 1, 2, 5, 10, 20: every cut, 23.4%, 30.7%, 41.0%, 46.9%, 49.0%; the deciles, 13.2%, 19.4%, 27.9%, 33.3%, 36.4%; the median, 4.4%, 6.9%, 11.3%, 14.4%, 16.2%.

Every variable, every cut

Slide the cut along every one of twenty baseline variables in a randomised trial of eighty whose treatment helps every patient equally, and some variable shows a Simpson reversal at some cut in 49.0% of trials — a coin toss with nothing to find. The twenty scans do not add like independent chances: if they did they would reverse 92.4% of the time. They read one patient risk between them, and the weak variables seldom reverse at all, so the strongest five supply 41.0 of the 49.0 points. A trial that shows one reversal usually shows several: 4.8 variables on average.

reversal · Simpson

Named alongside it

The objects these essays reach for when they reach for this one.

Effective number of testsMultiple comparisonsRandomisationSimpson's paradoxSubgroup analysisMedian splitPrognostic factorSpecification searchStratification

All concepts