Normal Distribution
nothing is called 95% until it has been counted
The distribution itself

The shape, and where its mass is

68, 95, 99.7 is recited more often than any other set of numbers in the subject. They are integrals of a specific curve, they are worth computing rather than remembering, and the third one is the one people misuse.
11 min read 6 figures Two routes to a numberThe tail is where it is read

The empirical rule — 68% within one standard deviation, 95% within two, 99.7% within three — is the most recited set of numbers in statistics. It is correct, it is worth knowing, and the way it is usually deployed contains a trap.

The normal density at sigma = 1.00The bands hold 68.27%, 95.45%, 99.73% of the mass. Those figures are integrals of the curve drawn, not the memorised 68-95-99.7.00.1000.2000.300-4-202standard deviations from the meandensity68.27%95.45%99.73%the bands are integrated, not recalledsigma = 1.30
Fig. 1 The normal density with the mass in each band computed by integrating the curve drawn, rather than recalled. Move the standard deviation and the curve changes shape; the percentages do not move at all.
8 exponential draws, standardised, against the normalThe source is one-sided and skewed. At n = 8 the standardised sum has skew 0.747, and the theory says 2/sqrt(n) = 0.707 — so the convergence is visible AND its rate is predicted.00.1000.2000.3000.400-2024standardised sumdensityskew 0.7472/√n = 0.70740,000 sums, one seed eachthe rate is predicted, not just the shape
Fig. 2 Where the shape comes from: standardised sums of something entirely unlike it.

What the numbers are

They are integrals. The mass within k standard deviations of the mean is

Φ(k)−Φ(−k)\Phi(k) - \Phi(-k)

and evaluating it gives 0.682689, 0.954500 and 0.997300. The figure computes these from the same distribution function every other page here uses, and asserts them against their known values to nine decimal places — so if the arithmetic behind any figure on this site drifted, this one would fail first.

The part the slider makes obvious

Move the standard deviation and watch the percentages.

They do not change. The curve gets wider or narrower, the peak rises or falls, and the mass inside one standard deviation stays at 68.27% throughout.

That is the point of the standard deviation as a unit: it is defined so that the proportions are the same for every normal distribution. A statement in sigmas is scale-free, which is what makes it comparable across quantities measured in entirely different units.

It is also why the three numbers are worth memorising at all. They are not facts about any particular dataset; they are facts about the shape.

Twenty samples of 40, every one of them genuinely normalEach panel is a quantile-quantile plot of 40 draws from a normal distribution. The worst point in the worst panel sits 1.52 standard deviations off the line. Anything a reader would reject here would be a false alarm.20 independent samples, 40 points eachall twenty are normalthis is what the noise looks like
Fig. 3 Twenty genuinely normal samples. Points well out in the tail are the expected furniture, not outliers.

The trap in the third number

99.7% sounds like almost everything, and the way it is usually stated encourages the reading “values beyond three sigma essentially do not happen.”

Turn it around. 0.27% is roughly one observation in 370. In a dataset of ten thousand, about twenty-seven observations should lie beyond three standard deviations. Finding some is not evidence of anything — finding none would be the surprising result.

This is the source of a common and expensive error: treating three-sigma observations as outliers to be removed. In a large enough sample they are the expected furniture of a normal distribution, and removing them makes the data less normal, not more, while shrinking the estimated variance and making every subsequent interval too narrow.

The normal approximation's error in the tail, n = 100, p = 0.05Approximate tail probability divided by the exact one. At one sigma the ratio is 0.98; at 4 sigma it is 0.104, so a rare event is understated by a factor of 10.1 sigma out0.982exact 1.3e-12 sigma out0.691exact 2.8e-23 sigma out0.335exact 4.3e-34 sigma out0.104exact 4.6e-41.0 would be exactapproximate ÷ exactunderstated where it matters most
Fig. 4 How wrong a tail probability can be when the distribution is only approximately normal.

And the trap in using it far out

The rule is quoted at one, two and three sigma. It gets used much further out, and that is where it stops being safe — for a reason that is about data rather than about arithmetic.

At six sigma the normal distribution puts about two in a billion beyond the limit. That number is arithmetically correct and empirically unverifiable: checking it would need a sample large enough to observe events at that rate, and a sample that size would make the model unnecessary.

What actually happens at six sigma is decided by whether the distribution really is normal that far out, and no sample can establish that. Real distributions depart from normality in the tails routinely — through mixtures, through occasional different mechanisms, through contamination — and the departures are invisible in the body of the data where all the observations are.

So a claim in the far tail is a claim about a modelling assumption, not a measurement. The same problem in a different guise is why an approximation that converges quickly in the middle is still badly wrong out there.

Where the normal approximation converges, and where it does notRelative error against the exact binomial. At n = 1280 the error at the median is 0.96% and three sigma out it is 25.7% — a factor of 27. The tail is where the approximation is used.sample sizerelative error10204080160320640128010%1%at the mediantwo sigma outthree sigma outexact binomial against its normal approximationthe tail converges last
Fig. 5 The middle and the tail converging at different rates, which is the same warning in a different form.
Student's t on 5 degrees of freedom, against the normalThe two-sided 95% critical value is 2.571 for t(5) and 1.960 for the normal — 31% wider. Using the normal at this sample size makes every interval too short by that much.00.1000.2000.3000.400-4-2024standard errors from the meandensityt 2.57z 1.96solid: t · dashed: normal31% wider at 5 df
Fig. 6 Two distributions that agree in the body and differ where it matters.

What is safe to take from it

A short list, since this is one of the few things in the subject worth carrying in the head.

One and two sigma are useful and robust. They cover the region where most data lives and where most distributions look alike.

Three sigma is a useful order of magnitude and a bad threshold. One in 370 is common in any dataset of size.

Beyond three sigma the number is model output rather than evidence. It can be computed, it cannot be checked, and it should be reported as an assumption.

And the general habit: when a proportion is quoted for a distribution, ask whether it was integrated or recalled. Here it is integrated, on every frame, which is why moving the slider does not move the number.

Why this shape and not another

The bands are properties of the curve, which raises the question of why this curve.

Three characterisations, each of which picks it out uniquely and none of which mentions the others.

It is the limit of standardised sums, which is the central limit theorem and the reason it appears in sampling distributions rather than in nature.

It is the maximum-entropy distribution for a given mean and variance. Among all distributions with a stated spread, it is the one assuming least else. That is why it is the honest default when the spread is the only thing known, and why assuming normality is a weaker assumption than it is usually accused of being.

It is the only distribution whose sample mean and sample variance are independent. That is a small, strange, exact property, and it is what makes the t distribution work — the denominator in a t statistic has to be independent of the numerator, and only the normal delivers that.

Three unrelated routes to the same curve, which is a fair reason for it to be over-represented in the subject even before the limit theorem is mentioned.

What the standard deviation is not

Two confusions that the band figure quietly settles.

It is not the average distance from the mean. That quantity is the mean absolute deviation, and for a normal distribution it is about 0.798 standard deviations — a different number, computed differently, with different properties. The standard deviation is the root of the mean squared deviation, and the squaring is why it is dominated by the far points rather than the typical ones.

It is not a percentile. One standard deviation happens to sit at the 84th percentile for a normal distribution and nowhere in particular for anything else. Quoting “one sigma” for a skewed distribution and expecting 68% inside is an error the bands make easy to see: the percentages in the figure are integrals of this curve and travel with it.

The bands under a different distribution

The figure holds sigma variable and the percentages fixed. Change the shape rather than the scale and the percentages move considerably.

For a t distribution on three degrees of freedom, the one-sigma band holds about 74% and the three-sigma band about 94% — noticeably different from 68% and 99.7%, and different in opposite directions, because the heavier tails pull mass out of the far band and the peak concentrates it in the near one.

For a uniform distribution the three-sigma band holds 100%, since the distribution simply stops.

That is the reason the empirical rule is stated for the normal and not for data in general. Applied to a distribution with heavier tails it overstates how much lies inside three sigma, and it overstates it exactly where a reader is most likely to be relying on it — in the tail.

Chebyshev’s floor, and why it is so weak

There is a version of the rule that holds for every distribution with a finite variance, and comparing it with the normal’s numbers is instructive.

Chebyshev’s inequality says at least 1 − 1/k² of the mass lies within k standard deviations. For k = 2 that is 75%, against the normal’s 95.45%. For k = 3 it is 88.9%, against 99.73%.

The gap is enormous, and it is not slack in the inequality — there are distributions that attain the bound. What it shows is how much the normal’s specific shape is doing: nearly all of the familiar tightness of the empirical rule comes from assuming normality, not from anything general about spread.

So a reader quoting 95% for two sigma is making a much stronger claim than they usually realise, and one that fails badly for a distribution with heavier tails than they assumed.

Where the constant comes from

The density carries a 1/√(2π) that looks arbitrary and is not, and knowing where it comes from makes the curve less of a formula to memorise.

The shape is e^(−x²/2); the constant is whatever makes it integrate to one. So the question is the value of the integral of e^(−x²/2) over the whole line, and the answer is √(2π).

The standard derivation is worth knowing because it is one of the neatest arguments in elementary mathematics: square the integral, read the product as a double integral over the plane, and change to polar coordinates, where it becomes elementary. The two-dimensionality is doing the work — the one-dimensional integral has no elementary antiderivative, and the two-dimensional version does.

That is also the reason the normal appears so naturally in two and three dimensions: a product of independent normals in Cartesian coordinates depends only on the radius, which is why velocity distributions and measurement errors in the plane take this form without anyone imposing it.

Reading a sigma figure in practice

The bands are a tool, and a short list of what they support makes the abstraction concrete.

A quality-control limit at three sigma expects one point in 370 outside by chance. On a chart with a point per hour, that is one false alarm every fortnight — which is the intended design and is routinely misread as a fault.

A “two-sigma event” in finance or climate means something in the outer 4.6%, which happens roughly one time in twenty-two. Describing it as rare overstates it considerably.

A measurement quoted as x ± σ covers 68%, not 95%. Reporting a one-sigma error bar and having it read as a 95% interval is one of the commonest miscommunications between fields, because different disciplines have different conventions and neither states which.

That last one is worth a habit: say what the bar is. “±1 s.e.” and “95% CI” are different objects differing by roughly a factor of two, and a figure carrying neither label cannot be read.

The two-dimensional version

The bands generalise, and the generalisation surprises people.

For a two-dimensional normal, the region within one sigma of the centre in both coordinates contains 0.683² ≈ 46.6% — not 68%. The probability inside an elliptical contour at “one sigma” is 39.3%.

The pattern continues: in higher dimensions almost all the probability sits in a thin shell away from the centre, not near it. In ten dimensions, the mode of the distribution of distances from the origin is near √10 σ, and the centre — which is still the highest-density point — is in a region of negligible volume.

That is the concentration-of-measure phenomenon, and it is why intuitions built on the one-dimensional bell are unreliable for anything multivariate. The familiar picture is a special case, and it is the special case with the least typical geometry.

Where this goes next

sample sizerelative error10204080160320640128010%1%at the mediantwo sigma outthree sigma outexact binomial against its normal approximationthe tail converges last The tail converges last The central limit theorem is usually shown as a shape arriving. What the demonstration leaves out is the rate — and the rate is wildly different in the middle and in the tail, which is where every approximation in the subject is actually read. 00.1000.2000.3000.400-2024standardised sumdensityskew 0.7472/√n = 0.70740,000 sums, one seed eachthe rate is predicted, not just the shape Sums of almost anything The theorem says sums converge on one shape whatever they are sums of, which is remarkable and true. Watching it happen from a one-sided skewed source, with the rate of convergence predicted in advance, is more convincing than watching the shape appear. 20 independent samples, 40 points eachall twenty are normalthis is what the noise looks like What normal actually looks like A single quantile plot of forty normal points wanders enough to look suspicious. Twenty of them, all genuinely normal, show what the noise looks like — and any single panel a reader would have rejected is in there.