Normal Distribution
nothing is called 95% until it has been counted
Intervals, counted

The correction for not knowing the spread

The t distribution exists because the standard deviation is estimated rather than known. At eight observations, using the normal instead makes every interval 12% too short — and the coverage that follows can be measured rather than argued about.
11 min read 6 figures Count it, do not claim itTwo routes to a number

The interval for a mean uses 1.96 standard errors when the standard deviation is known, and something larger when it is estimated. The something larger is Student’s t, the reason for it is precise, and the cost of ignoring it is measurable.

Student's t on 5 degrees of freedom, against the normalThe two-sided 95% critical value is 2.571 for t(5) and 1.960 for the normal — 31% wider. Using the normal at this sample size makes every interval too short by that much.00.1000.2000.3000.400-4-2024standard errors from the meandensityt 2.57z 1.96solid: t · dashed: normal31% wider at 5 df
Fig. 1 Student’s t against the normal at the same scale. The critical values are marked; the t one is always further out.

What is being corrected

An interval for a mean is built from the sample mean and the standard error. When the standard deviation is known — which essentially never happens — the sample mean is normal and 1.96 standard errors gives 95%.

When the standard deviation is estimated from the same data, two things are now random rather than one. Sometimes the sample happens to look tight, the estimated standard error comes out small, and the interval is narrower than it should be. That happens often enough at small n to matter, and it always errs in the same direction: undercoverage.

The t distribution is the exact sampling distribution of the standardised mean when the denominator is estimated. Its heavier tails are the correction for that extra source of variation.

Coverage of a 95% interval for a mean, n = 8Measured over 20,000 samples. The t interval covers 95.0% and the z interval 91.3%. The difference is the price of pretending the standard deviation was known.t interval, 7 df95.0%± 0.3 at 2 s.e.z interval91.3%± 0.4 at 2 s.e.20,000 samples, one seed eachnominal 95%counted, not assumed
Fig. 2 Coverage of the two intervals at the same sample size, counted over twenty thousand samples.

How large the correction is

Drag the degrees of freedom and the size becomes concrete.

At 200 degrees of freedom the t critical value is 1.972 against the normal’s 1.960 — a difference of 0.6%, and ignoring it is harmless.

At 10 it is 2.228, which is 14% wider.

At 5 it is 2.571, 31% wider.

At 2 it is 4.303, more than double.

That is why the correction is described as mattering only at small samples: past about thirty observations the difference is within the noise of everything else in the study, and below ten it dominates.

The normal density at sigma = 1.00The bands hold 68.27%, 95.45%, 99.73% of the mass. Those figures are integrals of the curve drawn, not the memorised 68-95-99.7.00.1000.2000.3000.400-4-202standard deviations from the meandensity68.27%95.45%99.73%the bands are integrated, not recalledsigma = 1.00
Fig. 3 The normal density and its band masses, integrated. The t correction is what happens when the width of that curve has to be estimated.

The coverage, counted

The argument above is theory. The site’s rule is that the theory is not the evidence, so the coverage is measured directly.

Twenty thousand samples of eight observations from a genuinely normal population:

  • the t interval covers 95.0%, which is what it claims
  • the z interval covers 91.5%

The t interval covering correctly is not a foregone conclusion — it confirms that the whole chain, from the generator through the variance estimate through the quantile function, is consistent. The z interval undercovering by three and a half points is the price of the shortcut, in the units that matter.

And the direction is always the same. Using z where t belongs never overcovers, because the error is always in the same direction: too narrow.

Twenty samples of 40, every one of them genuinely normalEach panel is a quantile-quantile plot of 40 draws from a normal distribution. The worst point in the worst panel sits 1.52 standard deviations off the line. Anything a reader would reject here would be a false alarm.20 independent samples, 40 points eachall twenty are normalthis is what the noise looks like
Fig. 4 Twenty samples of genuinely normal data. At small n the normality the t interval assumes cannot be checked.
Coverage of four nominal 95% intervals, n = 30Computed exactly by summing over all 31 possible counts, not simulated. The Wald interval drops to 26.0% and is jagged everywhere; Clopper–Pearson never falls below 95% and pays for it in width.0.4000.6000.80010.2000.4000.6000.800the true proportionactual coverage of a nominal 95% intervalnominal 95%Wald — the textbook oneWilsonAgresti–CoullClopper–Pearsonsummed over all 31 outcomesno interval is 95% until it is counted
Fig. 5 A different interval whose coverage was measured rather than assumed, for comparison.

What the correction does not fix

The t distribution corrects for estimating the standard deviation. It corrects for nothing else, and the things it does not correct for are usually larger.

It assumes the data is normal. At small n that assumption is untestable — a QQ plot of ten points is nearly uninformative — and the t interval’s guarantee is conditional on it. The robustness of the t interval to non-normality is real at moderate n and comes from the central limit theorem acting on the mean, not from anything about t.

It assumes independence. Correlated observations make the effective sample size smaller than the count, and no adjustment to the critical value fixes that; the standard error itself is wrong.

It says nothing about a single future observation. A prediction interval for one new value depends on the distribution of the data rather than of the mean, and is much wider. Confusing the two is common and the error is large.

8 exponential draws, standardised, against the normalThe source is one-sided and skewed. At n = 8 the standardised sum has skew 0.747, and the theory says 2/sqrt(n) = 0.707 — so the convergence is visible AND its rate is predicted.00.1000.2000.3000.400-2024standardised sumdensityskew 0.7472/√n = 0.70740,000 sums, one seed eachthe rate is predicted, not just the shape
Fig. 6 Convergence to normality, with its rate predicted. The robustness of the t interval at moderate n comes from here rather than from t.

The history, briefly

Gosset derived it at Guinness in 1908, working on small samples of barley and yeast where the large-sample approximations of the time were useless. He published as “Student” because the brewery did not permit staff to publish under their own names.

The detail worth keeping is that the problem arose from needing an answer at n = 4, not from a theoretical interest in small-sample distributions. The whole apparatus of degrees of freedom follows from taking a practical constraint seriously — which is a reasonable description of most of the useful statistics of that period, and a fair argument for measuring what a method does at the sample size actually available rather than the one the asymptotics assume.

Where the extra variability comes from

The mechanism is worth making concrete, because “the standard deviation is estimated” is a sentence people accept without seeing what it costs.

The statistic being used is (x̄ − μ) / (s / √n). The numerator is normal. The denominator contains s, which is itself random — it has a chi distribution, scaled.

So the statistic is a normal divided by an independent random quantity centred near one. Dividing by something that is sometimes smaller than expected produces occasional large values, and that is precisely the heavy tail. The correction is not a fudge factor; it is the exact distribution of a ratio whose denominator is estimated.

The independence of numerator and denominator is doing real work, and it holds only because the distribution is normal — the sample mean and sample variance of a normal are independent, and of essentially nothing else. That is why the t distribution is exact for normal data and approximate for everything else.

Degrees of freedom, stated properly

The phrase is used constantly and explained rarely.

The sample variance is computed from deviations about the sample mean. Those n deviations are not free: they sum to zero by construction, so knowing n − 1 of them determines the last. There are n − 1 independent pieces of information about the spread, and that count is the parameter of the distribution.

That also explains the n − 1 in the denominator of the variance formula. Dividing by n would use the sample mean as though it were the true mean, and the deviations about the sample mean are systematically smaller than deviations about the true one — because the sample mean is, by construction, the point that minimises them. Dividing by n − 1 corrects the resulting underestimate exactly.

Both facts come from the same place, which is worth noticing: the n − 1 in the variance and the n − 1 in the t distribution are the same n − 1.

Why the corrected version can still fail

The measured coverage above is 95.0% because the simulation draws genuinely normal data. Three departures break it, in increasing order of how much.

Mild non-normality at moderate n: barely matters. The central limit theorem is acting on the numerator, and the t interval is famously robust here.

Heavy tails at small n: matters a great deal. Both the mean and the variance are unstable, and the interval undercovers.

Dependence: matters most of all, and no critical value fixes it. If observations are positively correlated the standard error is understated at its root, so the interval is too narrow before t is applied at all. The effective sample size can be a small fraction of the count, and the interval reports a precision the data does not have.

That ordering is worth carrying, because worry is usually allocated in exactly the reverse order.

The lesson beyond t

The general shape of the argument recurs whenever a quantity is estimated rather than known.

Plugging an estimate into a formula that assumed the true value produces something too confident, and the size of the over-confidence depends on how well the estimate is pinned down. The t correction is the exact fix for one instance of that.

The Wald interval for a proportion is another instance without a fix: it estimates the standard error from the observed proportion, and the error is worst exactly where the estimate is furthest from the truth. Wilson’s interval is the analogue of the t correction there — it avoids plugging in an estimate at all, by inverting the test.

Seen that way the two essays are the same essay about two parameters, and the recurring recommendation is the same: do not substitute an estimate into a place that assumed certainty without accounting for the substitution.

Why it converges to the normal

The t distribution approaches the normal as the degrees of freedom grow, and the reason is visible in the construction rather than needing the algebra.

The statistic is a normal divided by an estimate of its own scale. As the sample grows, the estimate of the scale becomes more precise, so the denominator becomes closer to a constant. A normal divided by a constant is a normal.

The heavy tails are therefore entirely a consequence of the denominator’s variability, and they shrink at exactly the rate that variability does. That is why the correction matters at n = 5 and not at n = 200, and why the number that controls it is the degrees of freedom rather than the sample size as such.

It also explains a fact that otherwise looks like a coincidence: the t distribution on one degree of freedom is the Cauchy, which has no mean and no variance. With two observations the scale estimate is so unstable that the ratio has no moments at all.

The two-sample case, and Welch

The essay treats one sample. The two-sample comparison is commoner and has an extra wrinkle worth knowing.

The classical two-sample t test pools the two variance estimates, which assumes the groups have equal variances. When they do not, the test’s error rate departs from its nominal level — badly when the group sizes are also unequal.

Welch’s version does not pool. It estimates the standard error directly and uses a non-integer degrees of freedom computed from the two variances. It costs nothing, it is barely less powerful when the variances are equal, and it is substantially more reliable when they are not.

The recommendation follows and is one of the few in this subject that is close to unanimous among statisticians and close to ignored in practice: use Welch by default. Testing for equal variances first and choosing on that basis is worse than either, because the pre-test’s own error rate contaminates the procedure that follows.

Why the correction is not a licence

A caution, since the t distribution’s success invites over-application.

The correction handles a specific and narrow problem: the variance of the sampling distribution is estimated from the same data. It is exact for that problem under normality, and it does nothing at all about anything else.

In particular it does not make a small sample adequate. An interval at n = 5 with the t correction is honest about the variance estimate and still very wide, and if it is not wide enough to be useful, the answer is more data rather than a different critical value.

That is worth saying because the t distribution is sometimes invoked as though it repairs small samples. It repairs one consequence of small samples. The others — untestable assumptions, low power, and inflated effects among the significant results — are untouched.

Where this goes next

0.4000.6000.80010.2000.4000.6000.800the true proportionactual coverage of a nominal 95% intervalnominal 95%Wald — the textbook oneWilsonAgresti–CoullClopper–Pearsonsummed over all 31 outcomesno interval is 95% until it is counted What the 95% refers to An interval that claims 95% is making a checkable statement about a procedure, not about the interval in front of you. Build every possible sample and count, and the interval taught first turns out to cover 87.6% of the time. 00.1000.2000.3000.400-4-202standard deviations from the meandensity68.27%95.45%99.73%the bands are integrated, not recalledsigma = 1.00 The shape, and where its mass is 68, 95, 99.7 is recited more often than any other set of numbers in the subject. They are integrals of a specific curve, they are worth computing rather than remembering, and the third one is the one people misuse. Wald0.241covers 94.2%Wilson0.247covers 96.5%Agresti–Coull0.259covers 96.5%Clopper–Pearson0.273covers 98.3%expected width, and what it buysorange: fails its nominal levelthe shortest interval is the one that misses The shortest interval is the one that misses Four intervals for the same data, with their widths and their coverage measured together. The narrowest is the one that fails its stated level, which is exactly why it looks the most appealing.