Normal Distribution
nothing is called 95% until it has been counted
Concept

Breakdown point — where it appears

The smallest share of rows that, replaced by arbitrarily bad values, can move an estimate arbitrarily far. Least squares breaks at a single row; a trimmed fit only at about half, which is the most any estimator that treats rows alike can survive.

Named by 2 essays across one field — each of them below, with the objects they name alongside it.

Also named here as iteratively reweighted least squares, least trimmed squares, local minimum, m estimator, robust regression — the same set of essays touches all of them, so they are one junction rather than several.

One row at x = 9 on a wrong line, fitted three ways. Least squares gives slope −0.389, Huber 0.171 with the extra row at weight 0.115, and least trimmed squares 0.420, fitted to the 12 rows it keeps. The twenty clean rows alone give 0.495. Open circles are the rows the trimmed fit leaves out.

A robust loss and a far x

One far row drags least squares to a slope of −0.389. Huber's loss, the standard robust line, reaches only 0.171, and carried further out the same row gets its full weight back. Least trimmed squares reads 0.420 at every distance, and at the normal model keeps 7.13% of least squares' efficiency to do it.

regression · Leverage One row at x = 9 on a wrong line, fitted three ways. Least squares gives slope −0.389, Huber 0.171 with the extra row at weight 0.115, and least trimmed squares 0.420, fitted to the 12 rows it keeps. The MM-estimator carried on from the trimmed fit gives 0.479, with the extra row at weight 0.000 and the scale fixed at 0.319. The twenty clean rows alone give 0.495. Open circles are the rows the trimmed fit leaves out.

The start an efficient robust line inherits

The MM-estimator carries a trimmed fit on through a redescending loss, and it does what it promises on one far row: slope 0.479 at every distance, the row at weight exactly zero, and 87.2% of least squares' efficiency at twenty rows. What it cannot do is choose. At eight far rows of twenty the exact trimmed fit picks the wrong half on 111 datasets; the efficient step repairs none of them, spoils none of the other 89, and ends nearer the wrong line than the start did.

regression · Leverage

Named alongside it

The objects these essays reach for when they reach for this one.

Closed formEfficiencyIteratively reweighted least squaresLeast trimmed squaresLeverageLocal minimumM estimatorMaskingOutliersRobust regressionHuber lossInfluence function

All concepts