The same covariate, three ways round

Three worlds over a treatment, an outcome and a covariate, joined by the same three edges at the same three strengths — 0.90, 0.50 and 0.70 — differing only in which way the two edges touching the covariate point. In the first the covariate causes both and adjusting for it recovers the effect of 0.50 exactly. In the second the treatment causes the covariate, the effect is 1.13, and adjusting returns 0.50 — the direct edge alone, with the part that travels through the covariate deleted. In the third the treatment and the outcome both cause the covariate, the effect is 0.50, and adjusting returns -0.087. The regression that produces those three numbers is one formula, and nothing in the data says which panel it is being run in.

Three causal structures fitted to one covariance matrix over a treatment, a covariate and an outcome. Each reproduces it exactly — the largest entry-wise disagreement across all three is 4.4e-16 — so no sample of any size distinguishes them. The regression of the outcome on the treatment and the covariate returns 0.500 in all three, to within 4.4e-16, because that coefficient is a function of the covariance and of nothing else. The effect the three worlds hold is 0.500, 0.848 and 0.848: adjusting is exactly right in the first and off by −0.348 in the other two. The arithmetic cannot see the difference and the difference is the whole question.

What adjusting for every covariate measured does to the bias in the treatment's estimated effect, against adjusting for none, over 4000 randomly drawn structures of 6 covariates each. Each covariate is independently a common cause with probability 0.25, a cause of the treatment only, a cause of the outcome only, a cause of neither, a step on the causal path, or a common effect. The rule leaves a larger bias on 65.5% of structures, a smaller one on 33.8%, and the same on 0.7%. The share is a property of that population of structures rather than of adjustment, which is why the weights are stated; what does not depend on them is that the rule has no direction — it is not a conservative default that occasionally overcorrects, it is a rule whose error is whatever the structure happens to be.

A covariate measured before the treatment, caused by neither the treatment nor the outcome, and not a common cause of them. Two unmeasured variables sit behind it: one reaches the treatment, the other reaches the outcome, and both reach the covariate. Every rule of thumb for including a baseline variable is satisfied, and the regression that leaves the covariate out estimates the treatment's effect of 0.50 without bias, while the regression that includes it is off by −0.2000 — because the covariate is a common effect of the two unmeasured causes, and conditioning on a common effect makes its causes dependent. The path it opens runs from the treatment back through the first unmeasured cause, through the covariate, and out through the second to the outcome.

900 draws of two independent standard normal causes, with the 453 of them past a threshold of 0.00 marked and the 447 that fall short left pale. In the population the two are independent by construction. Inside the selected sample the correlation is -0.4669 in closed form and -0.5050 counted on these 453 rows, and the least-squares line through them has a slope of -0.545. The mechanism is visible in the picture rather than argued: the threshold removes one corner of the cloud, and a cloud with a corner missing is a cloud whose two coordinates carry information about each other.

Five quantities in a world where the treatment causes a covariate, the covariate causes the outcome, and an unmeasured variable causes both the covariate and the outcome. The treatment's total effect is 1.130 and its direct effect is 0.500. The regression that leaves the covariate out returns 1.130, which is the total effect exactly, because the unmeasured cause does not reach the treatment. The regression that includes it returns 0.050, counted at 0.048 over 500 fits of 600 rows — 1.080 below the total effect and 0.450 below the direct one. Remove the unmeasured cause and the same regression returns 0.500 exactly, so the number is not wrong because the covariate came after the treatment; it is wrong because conditioning on it opened a path that was closed.

The share of the confounding bias removed by adjusting for a proxy, against how well the proxy measures the confounder. The diagonal is the answer a reader would guess — a covariate that is 80% signal removes 80% of the problem. The curve is what the arithmetic gives: the reliability, times one minus the squared correlation between the treatment and the confounder, divided by one minus the product of those two. That squared correlation is 0.4475. A reliability of 0.8 removes 68.85% and one of 0.6 removes 45.32%. The two agree only at the ends, and the gap is widest where most applied covariates sit.