Normal Distribution
nothing is called 95% until it has been counted
Concept

Missing at random — where it appears

Missingness whose chance depends only on values that were recorded, not on the value that went missing. It is the assumption almost every standard repair rests on, and it is an assumption rather than a finding: the observed data cannot distinguish it from a mechanism that reads the missing value itself.

Named by 6 essays across 2 fields — each of them below, with the objects they name alongside it.

Also named here as missing not at random — the same set of essays touches all of them, so they are one junction rather than several.

Three mechanisms leave the slope alone; one does not. The bias of the complete-case slope under each of four missingness rules, counted over 4000 studies of 200 rows at 35.0% missing, with the closed form printed beside each count. Missingness that depends on nothing, on the regressor, or on the second covariate leaves the slope exactly where it was — the closed forms are zero to machine precision and the counts are -0.0005, -0.0005 and -0.0011 against standard errors of about 0.0018. Missingness that depends on the outcome moves it by -0.1635, which is 27.3% of the slope being estimated. The same share of rows is lost in every case.

Three mechanisms and one dataset

Four rules for which outcomes go missing, each calibrated to lose the same 35% of the rows and each leaning on what it reads with the same coefficient. Three leave the fitted slope exactly where it was, and the one that reads the outcome moves it by 0.163531.

missing · Missingness Unrepresentative in every respect but the one that matters. Three properties of the complete cases as the chance of being observed leans harder on the regressor, in closed form, at 35.0% of outcomes missing throughout. The mean of the regressor among the rows kept climbs from 0.0000 to 0.5528 against a population mean of zero, and the mean of the outcome from 0.0000 to 0.3980 above its own. The bias in the fitted slope is exactly zero at every one of the ten settings, because selection acting on the regressor alone leaves the conditional law of the outcome given the regressor untouched and least squares conditions on exactly that. The sample is wrong about almost everything and right about the one quantity being estimated.

Dropping the incomplete rows

Push the missingness until the rows that survive have a covariate mean of 0.543905 against a population zero and a variance of 0.5041 against one, and the fitted slope is still exactly right. Where the rule reads the outcome instead, the same sweep takes coverage to 2.42% at eight hundred rows.

missing · Missingness An estimate reported as a function of an assumption. What the slope really is, against a shift in the outcomes nobody saw — line from the closed form, dots counted over 2000 studies of 200 rows at 35.0% missing. Every point on this line produces exactly the same observed data, and the complete-case estimate is the flat line at 0.5996 regardless. The truth moves at -0.2845 per unit of shift, which is a function of the missingness model and the missing fraction and of nothing that can be estimated: across the swept range the true slope runs from 0.8845 to 0.3155, a span of 0.5691 against a value of 0.60 in the world where the shift is zero. Reporting the line is the honest form of the answer.

The mechanism the data cannot see

Two worlds produce identical covariates, identical patterns of what is recorded and identical recorded outcomes, to the last bit. Their true slopes are 0.6 and 0.315452, and the truth moves at 0.284548 per unit of an assumption nothing in the data can inform.

missing · Missingness A peak where the recorded data have none. The profile log-likelihood of a selection model in cy, the coefficient that lets the chance of being recorded depend on the outcome itself, for one study of 800 rows whose missingness is at random, with residuals normal; every other parameter is maximised at each fixed value. The model assumes the outcome is normal given the covariates. The curve peaks at cy = 0.35, where the fitted slope is 0.839, and the values of cy within the 95% cut run from −0.13 to 0.75; the likelihood-ratio statistic against cy = 0 is 1.47. The study was drawn with cy = 0.00. With the outcome's law left free, every value of cy fits the recorded rows equally well and this curve would be flat: its curvature is the normal assumption.

The assumption that identifies the mechanism

A selection model estimates how strongly an outcome decides whether it is recorded — the quantity two identical datasets showed no statistic can see — and it does so by assuming the outcome is normal. Where that holds and the outcome does decide, it repairs a slope complete cases put at 0.4318 to 0.5795. Where the missingness is at random and the residual is merely skewed, it reports selection that is not there, moves the slope from 0.5971 to 1.0319, and rejects missingness at random in 72.5% of studies.

missing · Missingness What the sixteen-run factorial's surviving runs estimate when about 1.3 runs are lost, by why they were lost. A 2⁴ factorial fitting eleven coefficients; true A = 1, B = 0.5, AB = 0.5, everything else 0, noise 1; 20,000 datasets. Runs are lost when the response exceeds 2.5, or at random at the high corner of A and B, or at random anywhere, each losing 1.26 runs on average. Bias of the fit on the surviving runs — lost at random: A 0.001, B 0.003, AB 0.000, C -0.002; lost at the high corner: A 0.003, B 0.001, AB 0.000, C -0.000; lost when the response passes the threshold: A -0.131, B -0.120, AB -0.117, C -0.000.

A run lost for a reason

A sixteen-run factorial that loses a run at random loses precision and nothing else. One that loses a run because the process failed — the response passed a threshold — loses about the same number of runs and biases exactly the effects that pushed the response there: an interaction of 0.5 reads 0.383, a main effect of 1 reads 0.869, and the interval around each still covers about 94%. Treating the lost runs as censored above the threshold overshoots with eleven coefficients and works with six, because a run's own leverage decides how much a censored observation pulls its own estimate.

design · Factorial How often a skewed study reports selection that is not there, by the strength of a variable that moves recording only. 120 studies of 800 rows at each strength, missing at random on the regressor with a right-skewed residual, 35% unrecorded. The variable w shifts the chance of being recorded and has no place in the outcome; its share of the recording index's variance is 0.0%, 9.3%, 29.1%, 62.1% at strengths 0, 0.5, 1, 2. The selection model that includes w rejects missingness at random on 75.8%, 27.5%, 7.5%, 5.0% of studies; the same model fitted without w rejects on 78.3%, 85.0%, 100.0%, 100.0%.

A variable that moves recording

A selection model fitted to a skewed outcome that is missing at random reports selection that is not there on three studies in four. Give it a variable that shifts who is recorded and has no place in the outcome — strong enough to carry 29% of the recording index's variance — and the false reports fall to 7.5%, the slope from 1.034 to 0.613 against a truth of 0.6. Leave the same variable out of the model and it does harm instead: false reports reach 100%, and where the outcome really does decide recording, the estimated pull falls from 1.18 to 0.78.

missing · Missingness

Named alongside it

The objects these essays reach for when they reach for this one.

Missing not at randomComplete-caseClosed formConfidence intervalMaximum likelihoodMissingness mechanismNon-identifiabilityObservation propensitySelection modelEstimandLeast squaresConditional distribution

All concepts