Every essay — page 3
Regression, and what the summary hides
A slope, a standard error and an R² can all be computed from data the model is grossly wrong about, and none of them says so. Leverage is a number known before the outcome is looked at, influence is a number, and whether a residual plot looks bad is a question with a calibrated answer. Two wrong rows placed together hide from every single-row diagnostic, and a loss that bounds a large residual does nothing about a far one.
The line that one point drew
A single observation among twenty-one reverses the sign of a fitted relationship. Its leverage is known from its x value before the outcome is looked at, so this is a property of the design rather than a surprise in the data.
Four datasets, one summary
Four datasets agree on slope, intercept and R² to two decimals. One is a linear relationship, one is a curve, one is a line with an outlier, and one has its slope set by a single point. The summary cannot tell them apart and neither can any other summary.
R² is not a measure of fit
Adding a predictor with no relationship to anything cannot reduce R², and in expectation raises it by 1/(n − 1). Twenty useless predictors on thirty points give an R² of 0.69 from pure noise.
Twenty residual plots
Judging whether a residual plot looks wrong requires knowing what a correct one looks like, and almost nobody has seen twenty of those. Here they are, from a model that is exactly right, at the sample size that matters.
Two points that hide each other
One far observation among twenty-one has a Cook's distance of 24.1. Put a second beside it and the two read 0.966 and 0.772, neither crossing 1, while together they reverse the slope and deleting both moves the fit by 53.3.
A robust loss and a far x
One far row drags least squares to a slope of −0.389. Huber's loss, the standard robust line, reaches only 0.171, and carried further out the same row gets its full weight back. Least trimmed squares reads 0.420 at every distance, and at the normal model keeps 7.13% of least squares' efficiency to do it.
The start an efficient robust line inherits
The MM-estimator carries a trimmed fit on through a redescending loss, and it does what it promises on one far row: slope 0.479 at every distance, the row at weight exactly zero, and 87.2% of least squares' efficiency at twenty rows. What it cannot do is choose. At eight far rows of twenty the exact trimmed fit picks the wrong half on 111 datasets; the efficient step repairs none of them, spoils none of the other 89, and ends nearer the wrong line than the start did.
The other half a trimmed fit rejected
The exact trimmed fit visits every half of the data, so it can report more than its winner: the best half whose line slopes the other way. Whenever the winner is reversed by a far cluster, that other half is the clean line. Printed beside the winner when its residual sum is within 1.70 times — a rule that raises five per cent of clean datasets — it flags 65% of the reversed fits at seven far rows of twenty, 55% at eight and 30% at nine, and no rule built on fit can do better at nine, where the far cluster has become the tighter half by every measure the data hold.
A standard error for a model that is wrong
Every standard error a regression prints is a statement about a model that is true, and the repair for a model that is not is one of the most-quoted lines in applied work. It is priced here rather than recommended: the robust error reads a spread the model-based one does not, and its 95% interval covers 88.73% at twenty rows, is the worse of the two under mild heteroskedasticity until fifty, and is the smaller of the two whenever the error variance sits in the middle of the design rather than at its edges. The four corrections differ in one number — what each does with a point's leverage — and on a design with a single far-out point that number takes one of them to 0.3191 of the truth and another to 5.1127 times it.
![One wrong model, four designs, four slopes. The slope a straight line converges to when the truth is a quadratic, under four covariate distributions, by two routes: the population projection in closed form, and the mean of 2500 fitted slopes at 200 rows apiece. The even spread over [0, 2] gives 1.6000 and the same spread moved to [1, 3] gives 2.6000, while widening it to [0, 4] gives 2.6000 — the same number as the shifted one, because a symmetric design's target is the truth's tangent slope at the design's own mean and does not read the spread at all. An exponential spread with the SAME mean as the first gives 2.6000. So two studies of one world, each fitting the same wrong model, honestly report slopes 1.0000 apart, and neither is making an error.](/f/one-wrong-model-four-designs-four-slopes-66577953.svg)
What a wrong model estimates
A straight line fitted to a curved truth converges on the tangent at its own design's mean. Two honest studies of one world, fitting the same wrong model, report 2.600000 and 1.600000, and neither is in error.
The bread and the filling
The robust standard error is not a safety margin. At one setting of the error variance it is 1.2806 times the model-based one and at another it is 0.8246 times it, and the sign of a single dial decides which.
Robust is not free
A robust standard error's promise is asymptotic and its use is not. Its 95% interval covers 88.73% at twenty rows, and under mild heteroskedasticity it is the worse of the two intervals until a hundred.
![What each correction is worth, exactly. Each variance estimate's expectation under a constant error variance, divided by the variance the slope actually has, at 20 rows on an even design, by two routes: the closed form E[eᵢ²] = σ²(1 − hᵢᵢ) carried through each correction's own weight, and the mean of 20000 counted estimates. The maximum leverage here is 0.1857 and the design's fourth-moment share Σu⁴/(Σu²)² is 0.0897, which is the only thing the closed form reads. HC0 comes out at 0.8603 — short by construction, since its factor is exactly 1 − 1/n − Σu⁴/(Σu²)². HC1 reaches 0.9559, HC2 is exactly 1.0000 at every design and every sample size, and HC3 overshoots to 1.1647. On an even design the four are within a fifth of each other and the choice barely matters.](/f/what-each-correction-is-worth-exactly-70392810.svg)
Three corrections and a leverage
On an even design of twenty rows the four robust corrections read 0.8603, 0.9559, 1.0000 and 1.1647 of the truth and the choice barely matters. Add one point at x = 8 and they read 0.3191, 0.3419, 1.0000 and 5.1127.
Right for the wrong reason
A robust standard error costs no coverage where the risk is absent — 95.52% against 95.06% at twenty rows. It costs a 6.89% wider interval and a variance estimate 2.572 times as variable, and the pre-test that would avoid paying recovers 15.9% of what the insurance is worth.
The count that is not the rows
Three hundred rows in five clusters of sixty carry 6.9000 times the variance an independent-rows calculation reports, and the interval that counts rows covers 53.42%. The same five unequal sizes laid out two ways give design effects of 9.3158 and 5.4652.
The reference the sandwich is read against
The cluster-robust interval covers 75.05% at five clusters and 93.58% at eighty. The same estimate read against a t on G − 2 covers 87.95% at five, and the estimator is unchanged — three hundred rows grouped into five clusters cover 74.28% where the same three hundred grouped into seventy-five cover 94.63%.
An effective number of clusters
Ten clusters of thirty rows, a covariate spread evenly across them, and a cluster-robust interval for the slope read against a t on G − 2 = 8 covers 91.1%. The slope needed about five degrees of freedom, not eight, because the two outermost clusters carry 49% of its variance and the middle two 0.6%. A count computed from the sizes and positions before any outcome is seen — 4.45 here, 3.32 when the big clusters sit at the edges — with the bias-reduced variance covers between 94.5% and 96.8% in all twelve layouts measured, where the usual interval on G − 2 covers between 76.2% and 94.5%.
A variable that moves one thing only
An instrument identifies a causal effect by assuming that a path no data can check is exactly zero, and it charges for that assumption in variance. Both prices are computed rather than described: a violation nobody can see is divided by the first stage, so the strength an instrument needs is 2.78 times the violation it is assumed not to have, and the conventional interval turns out to fail not when one instrument is weak but when many are — a failure each row's own first-stage leverage explains, and leaving that row out repairs at a price in width that depends on how strong the instruments are. What is left identified is the effect among the units the instrument actually moved, which at the standing compliance profile is 1.1000 against a population average of 0.5000.
The assumption nothing tests
An instrument buys a causal effect with an assumption no sample can check, and the price is set by the same quantity that made the method work. The first stage it needs is 2.7778 times the violation it is assumed not to have, so a direct effect of 0.05 demands a first stage of 0.1389 and least squares wins below it.
Weak, and back where it started
A consistent instrumental estimate at two hundred rows and a concentration parameter of 0.32 is biased by 0.3220 ± 0.0142 against a least-squares inconsistency of 0.3594 — 89.6% of the way back to the problem it was hired to solve. Just identified, it has no mean at all, and that is measured as a rate rather than assumed.
What the first stage does not know
A single weak instrument does not make the conventional interval undercover — it makes it cover 99.1% at a width of 7.320. Where the promise actually breaks is many instruments — coverage falls from 97.2% to 51.5% while the median width falls from 1.454 to 0.583.
Whose effect it is
With a perfectly valid instrument and no violation of anything, the estimate converges on 1.1000 where the population average effect is 0.5000. The gap is exactly θ(1 − p_c), the always-takers and never-takers cancel out of both halves of the ratio, and five per cent defiers move the answer to 1.2667.
Two instruments that disagree
The overidentification test keeps its size at 5.0% and reaches 86.4% power against a violation carried by one instrument. Against the same error carried by both in proportion to their first stages it rejects on 4.6% of draws — its own size — while the estimate is wrong by 0.3000, which is 94.2% of the confounding the instruments were brought in to remove.
Leaving each row out of its own first stage
Spread a fixed first-stage strength over thirty-two instruments and two-stage least squares covers 51.5%. Build each row's fitted treatment from a first stage that never saw that row and the same draws cover 98.7% — through an interval 5.99 times as wide, around an estimate that misses by more than the whole effect on 34.7% of draws. At eight times the strength the same repair covers 95.3% and costs a width factor of 1.66.
A standard error that knows about the instruments
Limited-information maximum likelihood came out least biased when a concentration parameter of 8 was spread over thirty-two instruments, and its conventional interval covered 79.0%. Bekker's many-instrument standard error covers 93.8% on the same draws, at 63% of the jackknife's width — and it gets there with a median standard error of 0.561 against a true spread of 0.797, because it is large on the draws that need it. At eight times the strength it covers 94.9% at 91% of the jackknife's width, and nothing measured here beats it.
Cells of unequal size and an error that follows them
Thirty-two instruments that are the indicators of cells holding 2 to 15 units each, and confounding whose variance is larger in the small cells. Limited-information maximum likelihood, the estimator that came out least biased with many instruments, then errs by 0.534 at a concentration of 8 — further than two-stage least squares — and Bekker's interval covers 74.8%. The jackknife version of the likelihood, which never lets a row's own leverage reach its own fitted value, errs by 0.066 and its robust interval covers 90.6%; at eight times the strength, 94.8% against Bekker's 87.3%. On normal instruments the same unequal variance costs almost nothing, because their leverages are nearly equal.