Normal Distribution
nothing is called 95% until it has been counted
Concept

Aggregation — where it appears

Combining groups into one comparison, which replaces each group's own weights with the pooled ones. A pooled rate is a weighted average of the group rates, so a comparison of pooled rates between differently-composed groups is not a comparison of the thing in question.

Named by 4 essays across 2 fields — each of them below, with the objects they name alongside it.

Which allocations reverse the overall comparison. The per-group success rates are held fixed; only the split of each group between treatment and control changes. 32% of the allocations reverse, and the worst reverses by 13.1 percentage points.

Simpson's reversal is a region, not a table

The treatment wins in both groups and loses overall. That is normally shown with one famous table, which cannot answer the two questions a reader has — how often, and how large. Swept, it turns out to occupy 31% of the allocation space.

paradox · Simpson How often a randomised trial reports the reversal, advantage 6 points. Simple randomisation against randomisation stratified by group, 4,000 trials at each size. The simple design reverses on 3.40% of trials at its worst size and 0.50% at 1280 units; the stratified design reverses on none of them, at any size.

The reversal a coin cannot prevent

Randomisation removes Simpson's reversal in expectation, which is not the same as removing it. A correctly randomised trial of eighty units, on a population where the treatment helps in both groups, reports it losing overall on 3.40% of trials — and stratifying the randomisation takes that to zero at every size.

reversal · Simpson One odds ratio in every stratum, and a different one marginally. Five strata with baseline risks from 5% to 85%, a treatment allocated by a coin in each, and a conditional odds ratio of exactly 2.5 throughout. The marginal odds ratio is 1.789. Nothing is confounded; the odds ratio is simply not a weighted average of odds ratios.

The change that is not confounding

Five strata, a treatment allocated by a coin in every one, and an odds ratio of exactly 2.5 in all five. The odds ratio computed on the pooled table is 1.789. Nothing is confounded — an odds ratio is not a weighted average of odds ratios, and the risk difference, on the same table, is exactly its own stratum value.

reversal · Simpson What stratifying on a mediator estimates, and what it does not. The direct effect stays at -0.109 throughout, because it is defined with the mediator held fixed. The total effect runs from -0.218 to 0.027 and crosses zero at 1.5. On 15% of the sweep the two have opposite signs, and both are correct answers.

Conditioning on what the treatment caused

When the grouping variable lies on the path from treatment to outcome, the stratified answer is the direct effect and the aggregate is the total effect. Both are correct. Over 15% of a sweep of the indirect path they have opposite signs, and no arithmetic on the table says which question was being asked.

reversal · Simpson

Named alongside it

The objects these essays reach for when they reach for this one.

ConfoundingRandomisationSimpson's paradoxAllocationStratificationBlockingCausal diagramCollapsibilityCovariate balanceDirect effectMediatorOdds ratio

All concepts