Normal Distribution
nothing is called 95% until it has been counted
Series

Summary — the series

5 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. Four datasets, slope 0.50, R² 0.67. Every one of these fits reports the same slope to two decimals and the same R². Only the first is a linear relationship with noise: the second is a curve, the third is a line with one outlier, and the fourth has its slope set by a single point.

    Four datasets, one summary

    Four datasets agree on slope, intercept and R² to two decimals. One is a linear relationship, one is a curve, one is a line with an outlier, and one has its slope set by a single point. The summary cannot tell them apart and neither can any other summary.

    part 1 · regression
  2. R² against the number of useless predictors, n = 30. The response is pure noise and so is every predictor, so the true relationship is nothing at all. R² rises from 0.000 to 0.648 anyway, following k/(n − 1) — which is what a criterion that rewards higher R² is actually rewarding.

    R² is not a measure of fit

    Adding a predictor with no relationship to anything cannot reduce R², and in expectation raises it by 1/(n − 1). Twenty useless predictors on thirty points give an R² of 0.69 from pure noise.

    part 2 · regression
  3. One relationship at five designs, residual spread 1.00. Every panel has the same slope of 1, the same intercept of 0 and the same residual standard deviation of 1.00. Only the range of x differs. R-squared runs from 0.021 to 0.849, and the estimated residual spread is 0.9932 in all five.

    R² is a property of the design

    One line, one residual spread, five studies that differ only in how far apart they placed their x values. R² runs from 0.021 to 0.849 and the estimated residual spread is 0.993 in every one of them. Nothing about the relationship changed.

    part 3 · spread
  4. Two studies with an R-squared of 0.85. The left study's points sit 0.50 from the line and the right study's 2.00 — a factor of 4.0. Both report an R-squared of 0.85 and, at the same sample size, the same standard error for the slope. The design was chosen to make it so, and it can always be chosen.

    The t statistic wearing different clothes

    For a simple regression, t² = (n − 2)R²/(1 − R²), exactly, on every dataset — checked to sixteen significant figures over five hundred fits. So a paper reporting R² and a p-value has reported one number twice, and two studies with the same R² have points four times further from the line.

    part 4 · spread
  5. How far apart four summaries put the quartet. Each bar is the largest value minus the smallest across the four datasets. Pearson spans 0.0018, which is the construction working. Distance correlation — the measure that is zero if and only if the variables are independent — spans 0.101, and Spearman spans 0.491.

    The summary that was meant to work

    Distance correlation is zero if and only if two variables are independent, which is exactly the guarantee a correlation coefficient lacks. Run on the four datasets that share a correlation, it spreads them by 0.10 — and Spearman, which guarantees nothing, spreads them by 0.49.

    part 5 · spread

All series