Concept

Observational equivalence — where it appears

The property of two models that imply exactly the same distribution of the observed variables. No amount of data separates them, so a quantity they disagree about is settled by assumption or by intervention rather than by a larger sample.

Named by 4 essays across 3 fields — each of them below, with the objects they name alongside it.

The first stage an instrument needs is set by the violation nobody can see. The error each estimator converges on when the instrument has a direct effect of 0.05 on the outcome — a path the exclusion restriction asserts is zero and no sample can check. The instrument's error is δ/π exactly, so it is the reciprocal of the very quantity that made the method work: 1.0000 at a first stage of 0.05 and 0.0833 at 0.60. Least squares carries the confounding instead, at 0.3440 at a first stage of 0.30. The two cross at π = 0.1389, and the crossing is exactly δ times 2.7778 — the first stage an instrument needs is proportional to the violation it is assumed not to have, and below that line the method being corrected is the better estimator.

The assumption nothing tests

An instrument buys a causal effect with an assumption no sample can check, and the price is set by the same quantity that made the method work. The first stage it needs is 2.7778 times the violation it is assumed not to have, so a direct effect of 0.05 demands a first stage of 0.1389 and least squares wins below it.

instrument · Exclusion

One distribution, two effects. Three causal structures fitted to one covariance matrix over a treatment, a covariate and an outcome. Each reproduces it exactly — the largest entry-wise disagreement across all three is 4.4e-16 — so no sample of any size distinguishes them. The regression of the outcome on the treatment and the covariate returns 0.500 in all three, to within 4.4e-16, because that coefficient is a function of the covariance and of nothing else. The effect the three worlds hold is 0.500, 0.848 and 0.848: adjusting is exactly right in the first and off by −0.348 in the other two. The arithmetic cannot see the difference and the difference is the whole question.

The two worlds that look the same

Three causal structures were fitted to one covariance matrix and agree with it to 4.4·10⁻¹⁶. The regression returns 0.5000 under all three; the effect they hold is 0.5000, 0.8481 and 0.8481. What separates structures is a missing edge, and the signature of one is a correlation of exactly zero.

collider · Conditioning

Two worlds, one Kaplan–Meier curve, two truths. World A gives each subject a frailty with mean one and variance 1, and multiplies both its event hazard (0.35) and its dropout hazard (0.5) by it, so the subjects likeliest to leave are the ones likeliest to fail. World B has independent event and dropout times whose hazards are world A's crude hazards. Kaplan–Meier over 1000 studies of 400 gives the same curve from both — 0.7750 and 0.7763 at t = 1; 0.6627 and 0.6641 at t = 2; 0.5924 and 0.5932 at t = 3; 0.5047 and 0.5042 at t = 5 — and that curve is world B's truth, 0.5052 at t = 5. World A's truth is 0.3636 there. The dashed lines are the two bounds that assume nothing, from every dropout failing on leaving (0.1905 at t = 5) to none ever failing (0.6667).

A dropout the data cannot see

Two worlds leave the same record to the last detail a study can write down — the same times, the same share ending in the event, the same share leaving first — and a log-rank test between them rejects at its own 5% level at every sample size from a hundred to sixteen hundred. Kaplan–Meier converges on 0.5052 at t = 5 from both. The truth is 0.5052 in one and 0.3636 in the other, and what is left to argue about is where between two bounds to stand.

survival · Censoring

The same error, caught or invisible, by how it is arranged. The overidentification test's rejection rate against the error the violation actually puts into the estimate, so the two rows are the same estimate being equally wrong. With the whole violation on one instrument the test keeps its size at 5.0% under the null and reaches 86.4% by an error of 0.800. With both instruments violating in the same ratio as their first stages the two Wald ratios are identical, the test has nothing to compare, and it rejects at 5.8% at that same error — its own size. Over 1000 draws of 300 rows at each setting, at a nominal 5.0%. The test is a comparison between instruments and it was never a check on either.

Two instruments that disagree

The overidentification test keeps its size at 5.0% and reaches 86.4% power against a violation carried by one instrument. Against the same error carried by both in proportion to their first stages it rejects on 4.6% of draws — its own size — while the estimate is wrong by 0.3000, which is 94.2% of the confounding the instruments were brought in to remove.

instrument · Exclusion

Named alongside it

The objects these essays reach for when they reach for this one.

Asymptotic bias Causal effect Exclusion restriction First stage Instrumental-variable Non-identifiability Null hypothesis Partial identification Statistical power Unmeasured confounding Wald ratio Causal diagram

All concepts