Normal Distribution
nothing is called 95% until it has been counted
Concept

Simpson's paradox — where it appears

A comparison that reverses when groups are pooled, with every number correct. It needs both a grouping variable related to the outcome and an unequal allocation across the arms, and it is arithmetically impossible when the arms have the same group composition.

Named by 3 essays across 2 fields — each of them below, with the objects they name alongside it.

Which allocations reverse the overall comparison. The per-group success rates are held fixed; only the split of each group between treatment and control changes. 32% of the allocations reverse, and the worst reverses by 13.1 percentage points.

Simpson's reversal is a region, not a table

The treatment wins in both groups and loses overall. That is normally shown with one famous table, which cannot answer the two questions a reader has — how often, and how large. Swept, it turns out to occupy 31% of the allocation space.

paradox · Simpson How often a randomised trial reports the reversal, advantage 6 points. Simple randomisation against randomisation stratified by group, 4,000 trials at each size. The simple design reverses on 3.40% of trials at its worst size and 0.50% at 1280 units; the stratified design reverses on none of them, at any size.

The reversal a coin cannot prevent

Randomisation removes Simpson's reversal in expectation, which is not the same as removing it. A correctly randomised trial of eighty units, on a population where the treatment helps in both groups, reports it losing overall on 3.40% of trials — and stratifying the randomisation takes that to zero at every size.

reversal · Simpson What stratifying on a mediator estimates, and what it does not. The direct effect stays at -0.109 throughout, because it is defined with the mediator held fixed. The total effect runs from -0.218 to 0.027 and crosses zero at 1.5. On 15% of the sweep the two have opposite signs, and both are correct answers.

Conditioning on what the treatment caused

When the grouping variable lies on the path from treatment to outcome, the stratified answer is the direct effect and the aggregate is the total effect. Both are correct. Over 15% of a sweep of the indirect path they have opposite signs, and no arithmetic on the table says which question was being asked.

reversal · Simpson

Named alongside it

The objects these essays reach for when they reach for this one.

AggregationAllocationConfoundingRandomisationBlockingCausal diagramCovariate balanceDirect effectMediatorSample sizeSampling variationStratification

All concepts