Concept

Student's t — where it appears

The distribution of a sample mean's departure from the population mean divided by the sample's own estimated standard error. It is wider than a normal at every degrees of freedom, and it is exact only because the mean and the variance of a normal sample are independent.

Named by 14 essays across 7 fields — each of them below, with the objects they name alongside it.

Which side a 95% t interval misses on, exponential source. Both tails should be 2.5%. At 8 observations the interval falls short of the mean on 9.75% of samples and overshoots on 0.31%. At 500 they are 3.31% and 2.05%, and the total is 5.36% — which a coverage table reports as very nearly right.

Where the two tails disagree

A 95% t interval on an exponential source at 120 observations covers 94.81%, which reads as very nearly right. It misses below the mean on 4.08% of samples and above on 1.11% — one tail 63% too heavy and the other 56% too light, and the total is the statistic that hides it.

tails · Student
The pooled two-sample test's size, with a true null everywhere. Forty units split between two groups, with the second group's variance a stated multiple of the first's, and the two population means equal. A 5% test should reject 5% of the time. The pooled test runs from 0.55% to 18.91% across this region; Welch's runs from 4.63% to 5.51%.

A degrees of freedom that is not a count

The pooled two-sample test's size runs from 0.55% to 18.91% across forty units split five ways against five variance ratios, with a true null in every cell. Welch's runs from 4.63% to 5.51% — bought with a degrees of freedom that is a function of the data, not an integer, and not a count of anything.

tails · Student
What the plot says, and what the interval does, at n = 40. For each source: how often a quantile plot of the data leaves its pointwise band, and how often the 95% t interval for the mean misses. The two-lump source leaves the band on 100% of samples and its interval covers 94.80%; the t on three degrees of freedom leaves it on 57% and covers 95.73%, the best of the five.

The plot is about the wrong quantity

A t interval needs the sampling distribution of the mean to be normal, not the data. A two-lump source leaves its quantile band on 100% of samples of forty and its interval covers 94.80%; a t on three degrees of freedom leaves it on 57% and covers 95.73%, the best of five sources.

lineup · Qq
Four 95% intervals for the mean of 30 exponential observations, split by the side they miss on. Both bars should read 2.5%. The t interval misses below the mean on 6.38% of samples. Widened until its total is exactly 5%, it misses below on 4.69% and above on 0.30%. Hall's transformation misses below on 3.31% and above on 1.96%.

The side a bound is read from

On thirty exponential observations the upper limit of a 95% t interval is exceeded by the true mean 6.38% of the time, against the 2.5% a safety margin set from it assumes. Widen the interval until its total coverage is exactly 95% and the upper limit is still exceeded 4.69% of the time. A symmetric repair fixes the number that is reported and not the one that is used; Hall's transformation, which bends the interval, takes the same rate to 3.31%.

tails · Student
Five upper limits for the mean of 15 observations from a exponential source, by margin and by failure rate. t limit: exceeded 8.51% at a mean margin of 0.523; Hall's transformation: exceeded 4.61% at a mean margin of 1.038; gamma family, shape known: exceeded 2.59% at a mean margin of 0.787; gamma family, shape by likelihood: exceeded 4.04% at a mean margin of 0.746; gamma family, shape by skewness: exceeded 17.51% at a mean margin of 0.421. The maximum-likelihood shape has a median of 1.10 and the skewness-based shape 3.20, against a true shape of 1.

A limit from the family it came from

On fifteen exponential observations an upper limit for the mean built from the gamma family's own exact pivot, with the shape estimated by maximum likelihood, is exceeded 4.04% of the time at a mean margin of 0.746 standard deviations. Hall's transformation is exceeded 4.61% at 1.038, and the same family limit with the shape read from the sample skewness is exceeded 17.51%. Knowing the family and estimating its one parameter well recovers most of what knowing the population buys — and the parameter is estimated well only from the logarithms.

tails · Student
The displacement per parameter, under five predictor laws. The out-of-sample differential one extra parameter costs a correct candidate, at a window of 80 rows, by two routes that share no arithmetic: counted from rolling forecasts over 1,500 draws of the fifteen-candidate table, and implied by the expected leverage of a fresh row over 80,000 draws of the predictors alone. The Gaussian closed form is 0.01386. normal: 0.01382 counted, 0.01386 from the leverage; Student's t, 8: 0.01379 counted, 0.01407 from the leverage; Student's t, 5: 0.01419 counted, 0.01448 from the leverage; Student's t, 3: 0.01751 counted, 0.01780 from the leverage; exponential: 0.01465 counted, 0.01487 from the leverage.

The displacement on a heavy tail

A correct candidate with one more parameter is behind out of sample by a closed-form amount, and the closed form rests on one Gaussian fact: the expected leverage of a fresh row is q/(R − q − 1). On predictors that are not Gaussian it becomes whatever the leverage is, and two routes that share no arithmetic agree on what that is. With a finite fourth moment it barely moves — 1.5% more per parameter at eight degrees of freedom, 4.5% at five, 7.3% on an exponential. At three degrees of freedom it is 28% more, and the estimate of it will not settle: four batches of twenty thousand draws disagree by more than a quarter.

select · Multiplicity
Student's t on 5 degrees of freedom, against the normal. The two-sided 95% critical value is 2.571 for t(5) and 1.960 for the normal — 31% wider. Using the normal at this sample size makes every interval too short by that much.

The correction for not knowing the spread

The t distribution exists because the standard deviation is estimated rather than known. At eight observations, using the normal instead makes every interval 12% too short — and the coverage that follows can be measured rather than argued about.

intervals · Student
Expected loss of three upper limits against the level each is set at, fifteen exponential observations, penalty 20. A failure costs 20 standard deviations of margin. At the conventional 97.5% the t limit's loss is 2.143, Hall's 1.922 and the fitted family's 1.530. Each at its own best level: the t limit 1.263 at 99.95%, Hall's 1.862 at 98.50%, the family 1.308 at 99.25%. The best fixed multiple of the standard error costs 1.255.

The level a limit should be set at

When a failure costs twenty times a standard deviation of margin, a t limit on fifteen exponential observations set at the conventional 97.5% has an expected loss of 2.143. Set at 99.95%, the same formula's loss is 1.263 — within 1% of the best any fixed multiple of s/√n can do, and ahead of both Hall's transformation and the fitted gamma family at their own best levels. Hall's is the worst at every level on every source. Choosing the level does more than choosing the construction, and the correction that reads the sample is the one no level rescues.

tails · Student
What a corrected critical value delivers to a t statistic: actual size over nominal, upper tail, 30 exponential draws. At a nominal 1% the upper-tail t test's actual size is ×0.153 with Student's quantile, ×2.94 with one Cornish–Fisher term and ×1.51 with Hall's cubic, both using the source's skewness, and ×0.878 with Hall's cubic using each sample's own. At 10⁻⁶ they are under a hundredth of nominal, ×1.04e+4, ×50.0 and ×1.59.

The correction a t test would use

On thirty exponential draws the one skewness term that held a sum's critical value within 10% out to five standard deviations nearly triples a t test's 1% size when the source's skewness is known, and its upper critical value turns back — at a level of 0.89% on ten draws, where a test asked for one in a thousand rejects 121 times too often. Hall's monotone cubic removes the turn and gains a floor. Built from the sample's own skewness it holds the short side within 20% from 10% down to one in a hundred thousand, and leaves the long side, the one a safety limit reads, at twice its level: the samples that fail are the ones that hide their skewness.

expansion · Rate
Expected loss of three upper limits against the level each is set at, when a failure is charged by its size, fifteen exponential observations, penalty 20. A failure costs 20 times the distance, in standard deviations, by which the mean exceeds the limit. At the conventional 97.5% the t limit's loss is 0.728, Hall's 1.136 and the fitted family's 0.835. Each at its own best level: the t limit 0.726 at 97.00%, Hall's 0.805 at 90.00%, the family 0.735 at 92.50%.

A failure charged by its size

Charge an upper limit's failure by how far the true mean exceeds it rather than by a fixed amount, and the level a limit should be set at falls further than any other change in this comparison moves it: on fifteen exponential observations, a penalty of twenty per standard deviation of shortfall asks for a t limit at 97% where a penalty of twenty per failure asked for 99.95%. The failures are small whichever limit produces them — about an eighth of a standard deviation — so a charge by size is a small charge per failure. The ranking hardly moves: the t limit stays within about 1% of the fitted family, whose failures are no smaller than its own, and Hall's transformation stays last by 11%.

tails · Student
A tilt for the ratio, read where it exists. The saddlepoint density of the draw's mean and mean square, integrated over the samples a tilt can reach that put the t statistic past its exact critical value, over the exact tail those same samples carry, at 30 exponential draws. On the short side the relative error runs from +0.88% at 10% to +2.47% at 10⁻⁵; on the long side from +0.83% to +2.05% at 10⁻⁶. Nothing in the construction was fitted to these tails; its error stays within 2.5% of the truth at every level shown.

A tilt for the ratio

A t statistic is a function of two means, so its saddlepoint tilts the joint law of each draw and its square. On an exponential source every such tilt is a truncated normal, and none has a coefficient of variation above one — the exponential is the edge of its own family. Integrated over the samples a tilt can reach, the saddlepoint reads a t statistic's tails on thirty draws within 2.5% from 10% to one in a million on the long side, with nothing fitted. The samples it cannot reach are a third of all samples, they sit at the centre of the statistic rather than in its tails, and a larger sample has more of them.

expansion · Rate
Approximate weights against the size of the run. What sixteen bands of the logarithms' t statistic are worth to a simulation of the lognormal t test's lower-tail size, in multiples of the draws, against the size of the run, with the band weights from Student's t exactly and from three approximations to it. Exact weights are worth 2.709 at every size. Fisher's one-term expansion is worth 2.708 at four thousand draws and 2.523 at a million, and breaks even with counting at 23.7 million draws. The variance-matched normal is worth 1.849 at four thousand and breaks even at 14,698. The standard normal is worth 0.762 at a thousand and breaks even at 669: a bias that does not shrink is multiplied by every draw added.

Weights from an approximate law

A companion whose law is only approximately known can still be cut into bands, with each band's weight computed from the approximation. The weight's error is a bias that does not shrink, and the run multiplies it: the stratified rate is worth p(1 − p)/(v + Nb²) times the draws. Reading Student's t on nineteen degrees of freedom as normal breaks even at 669 draws; the normal with the t's variance at about fifteen thousand; Fisher's one-term expansion at twenty-four million. What decides the bias is how the weight errors line up with the band rates, not how large they are — Fisher's is wrong by sixty per cent in one band and biases the rate by six hundredths of one.

splits · Routes
What it costs to set the t limit at one level without knowing which skewed source the 15 observations came from. For the t limit on 15 observations, charged twenty per standard deviation of shortfall plus its margin, the loss at each nominal level as an excess over the best level for each source. The best levels are 97% for the exponential, 96.25% for the gamma, shape 2, 97.75% for the gamma, shape ½, 98.25% for the lognormal. The minimax level, 97.25%, costs at most 0.8% on any of them: 0.1% on the exponential, 0.6% on the gamma, shape 2, 0.2% on the gamma, shape ½, 0.8% on the lognormal. The conventional 97.5% costs up to 1.0%.

A level set before the source is known

Every best level for an upper limit on skewed data was found knowing the source, and the fifteen observations that would tell the sources apart are the ones being used to set the limit. Chosen instead to protect the worst of four skewed sources, the t limit's level is 97.25% when a failure is charged by the size of its shortfall, and it costs at most 0.8% more than each source's own best — the sources disagree about the best level and agree that missing it a little is cheap. The fitted gamma family, the construction meant to adapt to the source, is the one that needs it: its best levels run from 90% to 96.5% and its minimax level costs 3.4%. Charged a fixed amount per failure, the t limit's minimax level is 99.97% and costs 6.5%; the conventional 97.5% costs up to 119.9%.

tails · Student
Coverage of the population mean from two units, by the true spread between units. 4,000 two-unit studies at each true spread, ten observations a unit, within-unit standard deviation 1.2. exact, on one degree of freedom: 94.97%, 95.50%, 94.95%, 94.83%, 95.15%, 95.20%, 94.67%. spread supplied as 1: 100.00%, 100.00%, 99.92%, 95.53%, 81.85%, 70.03%, 51.55%. half-normal, scale 1: 100.00%, 100.00%, 99.78%, 94.63%, 86.25%, 79.70%, 68.38%. half-Cauchy, scale 1: 100.00%, 100.00%, 100.00%, 98.98%, 95.50%, 92.38%, 88.00%. inverse-gamma(0.001, 0.001) on the variance: 100.00%, 100.00%, 100.00%, 99.15%, 97.05%, 95.28%, 92.65% — at spreads 0, 0.25, 0.5, 1, 1.5, 2, 3. The exact interval holds 95% at every spread; every integrated one over-covers when the spread is small and under-covers when it is large, and where it crosses 95% is set by the prior's tail.

A prior on the spread between two sites

Integrate the between-site spread of a two-site study under a half-normal prior of scale 1 and the interval for the overall mean is 3.01 wide and covers 94.63% when the true spread is 1 — the supplied interval again, under another name. At a true spread of 3 it covers 68.38%. Every prior tried covers 95% averaged over its own draws and none covers it at every spread; each holds up to about its own scale, because two sites move the posterior median of the spread by less than a third of what the truth moves.

multilevel · Levels

Named alongside it

The objects these essays reach for when they reach for this one.

SkewnessCoverageSample sizeUpper confidence limitEstimated varianceDecision theoryDegrees of freedomEdgeworth expansionExpected lossMonte CarloNormalitySignificance level

All concepts