Student — the series
-
The correction for not knowing the spread
The t distribution exists because the standard deviation is estimated rather than known. At eight observations, using the normal instead makes every interval 12% too short — and the coverage that follows can be measured rather than argued about.
-
Where the two tails disagree
A 95% t interval on an exponential source at 120 observations covers 94.81%, which reads as very nearly right. It misses below the mean on 4.08% of samples and above on 1.11% — one tail 63% too heavy and the other 56% too light, and the total is the statistic that hides it.
-
A degrees of freedom that is not a count
The pooled two-sample test's size runs from 0.55% to 18.91% across forty units split five ways against five variance ratios, with a true null in every cell. Welch's runs from 4.63% to 5.51% — bought with a degrees of freedom that is a function of the data, not an integer, and not a count of anything.
-
The skewness of a difference
Welch's test holds its size to within half a point when both groups are normal. Give both groups the same skewed population and it still balances at twenty and twenty — and at eight and thirty-two it rejects low on 7.16% of samples and high on 0.66%. One number decides which: the skewness of the difference of the two means, which ranks twenty-five cells by their imbalance with a correlation of 0.997.
-
The side a bound is read from
On thirty exponential observations the upper limit of a 95% t interval is exceeded by the true mean 6.38% of the time, against the 2.5% a safety margin set from it assumes. Widen the interval until its total coverage is exactly 95% and the upper limit is still exceeded 4.69% of the time. A symmetric repair fixes the number that is reported and not the one that is used; Hall's transformation, which bends the interval, takes the same rate to 3.31%.
-
A limit from the family it came from
On fifteen exponential observations an upper limit for the mean built from the gamma family's own exact pivot, with the shape estimated by maximum likelihood, is exceeded 4.04% of the time at a mean margin of 0.746 standard deviations. Hall's transformation is exceeded 4.61% at 1.038, and the same family limit with the shape read from the sample skewness is exceeded 17.51%. Knowing the family and estimating its one parameter well recovers most of what knowing the population buys — and the parameter is estimated well only from the logarithms.
-
The level a limit should be set at
When a failure costs twenty times a standard deviation of margin, a t limit on fifteen exponential observations set at the conventional 97.5% has an expected loss of 2.143. Set at 99.95%, the same formula's loss is 1.263 — within 1% of the best any fixed multiple of s/√n can do, and ahead of both Hall's transformation and the fitted gamma family at their own best levels. Hall's is the worst at every level on every source. Choosing the level does more than choosing the construction, and the correction that reads the sample is the one no level rescues.
-
A failure charged by its size
Charge an upper limit's failure by how far the true mean exceeds it rather than by a fixed amount, and the level a limit should be set at falls further than any other change in this comparison moves it: on fifteen exponential observations, a penalty of twenty per standard deviation of shortfall asks for a t limit at 97% where a penalty of twenty per failure asked for 99.95%. The failures are small whichever limit produces them — about an eighth of a standard deviation — so a charge by size is a small charge per failure. The ranking hardly moves: the t limit stays within about 1% of the fitted family, whose failures are no smaller than its own, and Hall's transformation stays last by 11%.
-
A level set before the source is known
Every best level for an upper limit on skewed data was found knowing the source, and the fifteen observations that would tell the sources apart are the ones being used to set the limit. Chosen instead to protect the worst of four skewed sources, the t limit's level is 97.25% when a failure is charged by the size of its shortfall, and it costs at most 0.8% more than each source's own best — the sources disagree about the best level and agree that missing it a little is cheap. The fitted gamma family, the construction meant to adapt to the source, is the one that needs it: its best levels run from 90% to 96.5% and its minimax level costs 3.4%. Charged a fixed amount per failure, the t limit's minimax level is 99.97% and costs 6.5%; the conventional 97.5% costs up to 119.9%.