Normal Distribution
nothing is called 95% until it has been counted
Concept

Selection model — where it appears

A joint model for an outcome and for whether that outcome is recorded, with the two linked through variables they share. Writing the mechanism down this way makes the bias of an analysis that ignores it a closed expression rather than something to be simulated.

Named by 4 essays across one field — each of them below, with the objects they name alongside it.

Three mechanisms leave the slope alone; one does not. The bias of the complete-case slope under each of four missingness rules, counted over 4000 studies of 200 rows at 35.0% missing, with the closed form printed beside each count. Missingness that depends on nothing, on the regressor, or on the second covariate leaves the slope exactly where it was — the closed forms are zero to machine precision and the counts are -0.0005, -0.0005 and -0.0011 against standard errors of about 0.0018. Missingness that depends on the outcome moves it by -0.1635, which is 27.3% of the slope being estimated. The same share of rows is lost in every case.

Three mechanisms and one dataset

Four rules for which outcomes go missing, each calibrated to lose the same 35% of the rows and each leaning on what it reads with the same coefficient. Three leave the fitted slope exactly where it was, and the one that reads the outcome moves it by 0.163531.

missing · Missingness An estimate reported as a function of an assumption. What the slope really is, against a shift in the outcomes nobody saw — line from the closed form, dots counted over 2000 studies of 200 rows at 35.0% missing. Every point on this line produces exactly the same observed data, and the complete-case estimate is the flat line at 0.5996 regardless. The truth moves at -0.2845 per unit of shift, which is a function of the missingness model and the missing fraction and of nothing that can be estimated: across the swept range the true slope runs from 0.8845 to 0.3155, a span of 0.5691 against a value of 0.60 in the world where the shift is zero. Reporting the line is the honest form of the answer.

The mechanism the data cannot see

Two worlds produce identical covariates, identical patterns of what is recorded and identical recorded outcomes, to the last bit. Their true slopes are 0.6 and 0.315452, and the truth moves at 0.284548 per unit of an assumption nothing in the data can inform.

missing · Missingness A peak where the recorded data have none. The profile log-likelihood of a selection model in cy, the coefficient that lets the chance of being recorded depend on the outcome itself, for one study of 800 rows whose missingness is at random, with residuals normal; every other parameter is maximised at each fixed value. The model assumes the outcome is normal given the covariates. The curve peaks at cy = 0.35, where the fitted slope is 0.839, and the values of cy within the 95% cut run from −0.13 to 0.75; the likelihood-ratio statistic against cy = 0 is 1.47. The study was drawn with cy = 0.00. With the outcome's law left free, every value of cy fits the recorded rows equally well and this curve would be flat: its curvature is the normal assumption.

The assumption that identifies the mechanism

A selection model estimates how strongly an outcome decides whether it is recorded — the quantity two identical datasets showed no statistic can see — and it does so by assuming the outcome is normal. Where that holds and the outcome does decide, it repairs a slope complete cases put at 0.4318 to 0.5795. Where the missingness is at random and the residual is merely skewed, it reports selection that is not there, moves the slope from 0.5971 to 1.0319, and rejects missingness at random in 72.5% of studies.

missing · Missingness How often a skewed study reports selection that is not there, by the strength of a variable that moves recording only. 120 studies of 800 rows at each strength, missing at random on the regressor with a right-skewed residual, 35% unrecorded. The variable w shifts the chance of being recorded and has no place in the outcome; its share of the recording index's variance is 0.0%, 9.3%, 29.1%, 62.1% at strengths 0, 0.5, 1, 2. The selection model that includes w rejects missingness at random on 75.8%, 27.5%, 7.5%, 5.0% of studies; the same model fitted without w rejects on 78.3%, 85.0%, 100.0%, 100.0%.

A variable that moves recording

A selection model fitted to a skewed outcome that is missing at random reports selection that is not there on three studies in four. Give it a variable that shifts who is recorded and has no place in the outcome — strong enough to carry 29% of the recording index's variance — and the false reports fall to 7.5%, the slope from 1.034 to 0.613 against a truth of 0.6. Leave the same variable out of the model and it does harm instead: false reports reach 100%, and where the outcome really does decide recording, the estimated pull falls from 1.18 to 0.78.

missing · Missingness

Named alongside it

The objects these essays reach for when they reach for this one.

Complete-caseMissing at randomMissing not at randomNon-identifiabilityClosed formConfidence intervalMaximum likelihoodMissingness mechanismObservation propensityEstimandLikelihood ratio testModel misspecification

All concepts