The shape, and where its mass is

The empirical rule — 68% within one standard deviation, 95% within two, 99.7% within three — is the most recited set of numbers in statistics. It is correct, it is worth knowing, and the way it is usually deployed contains a trap.

What the numbers are #

They are integrals. The mass within k standard deviations of the mean is

$$\Phi(k) - \Phi(-k)$$

and evaluating it gives 0.682689, 0.954500 and 0.997300. The figure computes these from the same distribution function every other page here uses, and asserts them against their known values to nine decimal places — so if the arithmetic behind any figure on this site drifted, this one would fail first.

Two and 1.96 are not the same number #

One distinction is worth drawing before the figures are used for anything, because the two quantities are routinely quoted as though they were one.

Integrating the curve gives 68.27% within one standard deviation, 95.45% within two and 99.73% within three.

The 95% that appears in every interval on this site is not the two-standard-deviation figure. It is the mass within 1.96 standard deviations, and 1.96 is the answer to a different question — how far out the interval must reach to enclose 95% rather than how much is enclosed by two.

The gap is 0.45 percentage points, which is small and is not nothing: an interval built at two standard deviations is a 95.45% interval, and one built at 1.96 is a 95% interval, and only the second is the one being promised.

Further out the masses fall away fast. Beyond three standard deviations there is 0.27%; beyond four, 0.0063%; beyond five, 0.000057%. Each additional standard deviation removes between forty and a hundred times what is left, which is why the tail figures are quoted as orders of magnitude rather than as percentages.

The part the slider makes obvious #

Move the standard deviation and watch the percentages.

They do not change. The curve gets wider or narrower, the peak rises or falls, and the mass inside one standard deviation stays at 68.27% throughout.

That is the point of the standard deviation as a unit: it is defined so that the proportions are the same for every normal distribution. A statement in sigmas is scale-free, which is what makes it comparable across quantities measured in entirely different units.

It is also why the three numbers are worth memorising at all. They are not facts about any particular dataset; they are facts about the shape.

The trap in the third number #

99.7% sounds like almost everything, and the way it is usually stated encourages the reading “values beyond three sigma essentially do not happen.”

Turn it around. 0.27% is roughly one observation in 370. In a dataset of ten thousand, about twenty-seven observations should lie beyond three standard deviations. Finding some is not evidence of anything — finding none would be the surprising result.

This is the source of a common and expensive error: treating three-sigma observations as outliers to be removed. In a large enough sample they are the expected furniture of a normal distribution, and removing them makes the data less normal, not more, while shrinking the estimated variance and making every subsequent interval too narrow.

And the trap in using it far out #

The rule is quoted at one, two and three sigma. It gets used much further out, and that is where it stops being safe — for a reason that is about data rather than about arithmetic.

At six sigma the normal distribution puts about two in a billion beyond the limit. That number is arithmetically correct and empirically unverifiable: checking it would need a sample large enough to observe events at that rate, and a sample that size would make the model unnecessary.

What actually happens at six sigma is decided by whether the distribution really is normal that far out, and no sample can establish that. Real distributions depart from normality in the tails routinely — through mixtures, through occasional different mechanisms, through contamination — and the departures are invisible in the body of the data where all the observations are.

So a claim in the far tail is a claim about a modelling assumption, not a measurement. The same problem in a different guise is why an approximation that converges quickly in the middle is still badly wrong out there.

What is safe to take from it #

A short list, since this is one of the few things in the subject worth carrying in the head.

One and two sigma are useful and robust. They cover the region where most data lives and where most distributions look alike.

Three sigma is a useful order of magnitude and a bad threshold. One in 370 is common in any dataset of size.

Beyond three sigma the number is model output rather than evidence. It can be computed, it cannot be checked, and it should be reported as an assumption.

And the general habit: when a proportion is quoted for a distribution, ask whether it was integrated or recalled. Here it is integrated, on every frame, which is why moving the slider does not move the number.

Why this shape and not another #

The bands are properties of the curve, which raises the question of why this curve.

Three characterisations, each of which picks it out uniquely and none of which mentions the others.

It is the limit of standardised sums, which is the central limit theorem and the reason it appears in sampling distributions rather than in nature.

It is the maximum-entropy distribution for a given mean and variance. Among all distributions with a stated spread, it is the one assuming least else. That is why it is the honest default when the spread is the only thing known, and why assuming normality is a weaker assumption than it is usually accused of being.

It is the only distribution whose sample mean and sample variance are independent. That is a small, strange, exact property, and it is what makes the t distribution work — the denominator in a t statistic has to be independent of the numerator, and only the normal delivers that.

Three unrelated routes to the same curve, which is a fair reason for it to be over-represented in the subject even before the limit theorem is mentioned.

What the standard deviation is not #

Two confusions that the band figure quietly settles.

It is not the average distance from the mean. That quantity is the mean absolute deviation, and for a normal distribution it is about 0.798 standard deviations — a different number, computed differently, with different properties. The standard deviation is the root of the mean squared deviation, and the squaring is why it is dominated by the far points rather than the typical ones.

It is not a percentile. One standard deviation happens to sit at the 84th percentile for a normal distribution and nowhere in particular for anything else. Quoting “one sigma” for a skewed distribution and expecting 68% inside is an error the bands make easy to see: the percentages in the figure are integrals of this curve and travel with it.

The bands under a different distribution #

The figure holds sigma variable and the percentages fixed. Change the shape rather than the scale and the percentages move considerably.

For a t distribution on three degrees of freedom, the one-sigma band holds about 74% and the three-sigma band about 94% — noticeably different from 68% and 99.7%, and different in opposite directions, because the heavier tails pull mass out of the far band and the peak concentrates it in the near one.

For a uniform distribution the three-sigma band holds 100%, since the distribution simply stops.

That is the reason the empirical rule is stated for the normal and not for data in general. Applied to a distribution with heavier tails it overstates how much lies inside three sigma, and it overstates it exactly where a reader is most likely to be relying on it — in the tail.

Chebyshev’s floor, and why it is so weak #

There is a version of the rule that holds for every distribution with a finite variance, and comparing it with the normal’s numbers is instructive.

Chebyshev’s inequality says at least 1 − 1/k² of the mass lies within k standard deviations. For k = 2 that is 75%, against the normal’s 95.45%. For k = 3 it is 88.9%, against 99.73%.

The gap is enormous, and it is not slack in the inequality — there are distributions that attain the bound. What it shows is how much the normal’s specific shape is doing: nearly all of the familiar tightness of the empirical rule comes from assuming normality, not from anything general about spread.

So a reader quoting 95% for two sigma is making a much stronger claim than they usually realise, and one that fails badly for a distribution with heavier tails than they assumed.

Where the constant comes from #

The density carries a 1/√(2π) that looks arbitrary and is not, and knowing where it comes from makes the curve less of a formula to memorise.

The shape is $e^{-x^2/2}$; the constant is whatever makes it integrate to one. So the question is the value of the integral of $e^{-x^2/2}$ over the whole line, and the answer is √(2π).

The standard derivation is worth knowing because it is one of the neatest arguments in elementary mathematics: square the integral, read the product as a double integral over the plane, and change to polar coordinates, where it becomes elementary. The two-dimensionality is doing the work — the one-dimensional integral has no elementary antiderivative, and the two-dimensional version does.

That is also the reason the normal appears so naturally in two and three dimensions: a product of independent normals in Cartesian coordinates depends only on the radius, which is why velocity distributions and measurement errors in the plane take this form without anyone imposing it.

Reading a sigma figure in practice #

The bands are a tool, and a short list of what they support makes the abstraction concrete.

A quality-control limit at three sigma expects one point in 370 outside by chance. On a chart with a point per hour, that is one false alarm every fortnight — which is the intended design and is routinely misread as a fault.

A “two-sigma event” in finance or climate means something in the outer 4.6%, which happens roughly one time in twenty-two. Describing it as rare overstates it considerably.

A measurement quoted as x ± σ covers 68%, not 95%. Reporting a one-sigma error bar and having it read as a 95% interval is one of the commonest miscommunications between fields, because different disciplines have different conventions and neither states which.

That last one is worth a habit: say what the bar is. “±1 s.e.” and “95% CI” are different objects differing by roughly a factor of two, and a figure carrying neither label cannot be read.

The two-dimensional version #

The bands generalise, and the generalisation surprises people.

For a two-dimensional normal, the region within one sigma of the centre in both coordinates contains 0.683² ≈ 46.6% — not 68%. The probability inside an elliptical contour at “one sigma” is 39.3%.

The pattern continues: in higher dimensions almost all the probability sits in a thin shell away from the centre, not near it. In ten dimensions, the mode of the distribution of distances from the origin is near √10 σ, and the centre — which is still the highest-density point — is in a region of negligible volume.

That is the concentration-of-measure phenomenon, and it is why intuitions built on the one-dimensional bell are unreliable for anything multivariate. The familiar picture is a special case, and it is the special case with the least typical geometry.

The same bands under a different shape #

The three numbers belong to the normal distribution and are quoted as though they belonged to data. Applying them to something that is not normal is the commonest way they mislead, and the size of the error is worth having rather than guessing.

Taking a t distribution — the same symmetric bell shape with heavier tails — and asking what fraction of its mass lies within one, two and three of its own standard deviations:

• t with 3 degrees of freedom: 81.8%, 96.0%, 98.6%
• t with 5: 74.7%, 95.1%, 98.8%
• t with 10: 71.0%, 95.1%, 99.3%
• t with 30: 69.1%, 95.3%, 99.6%
• normal: 68.3%, 95.4%, 99.7%

Two features are worth noticing, and the second is the one that causes trouble.

The two-sigma number is remarkably stable. Every row is within half a point of 95%, even the one with tails heavy enough that the fourth moment does not exist. Anyone using two sigma as a rough 95% interval is on firmer ground than they have any right to be.

The three-sigma number is not stable, and it fails in the dangerous direction. Under t with 3 degrees of freedom, 1.4% of the mass lies beyond three standard deviations against the normal’s 0.27% — five times as much. Someone treating a three-sigma event as a 1-in-370 occurrence when the data has heavy tails is understating its frequency fivefold, and the understatement grows as the band is pushed further out.

A uniform distribution makes the opposite point. Its mass within one standard deviation is 57.7%, and within two standard deviations it is 100% exactly — there is nothing beyond 1.73 standard deviations, because the distribution has an edge. The three numbers are not even approximately right, and no tail risk exists at all.

So the bands are a property of one specific shape. They are a good approximation for many shapes at two sigma, and a poor one for heavy-tailed shapes at three and beyond, which is precisely the region where they get quoted in risk arguments.

What is guaranteed regardless of shape #

There is a bound that holds for every distribution with a finite variance, and its weakness is instructive.

Chebyshev’s inequality says at least 1 − 1/k² of the mass lies within k standard deviations. That gives:

• within 1 sigma: nothing at all — the bound is vacuous
• within 2 sigma: at least 75%
• within 3 sigma: at least 88.9%

Set against the normal’s 95.4% and 99.7%, the bound is very weak indeed. At three sigma it permits eleven per cent of the mass outside, where the normal has less than a third of one per cent — a factor of forty.

That weakness is the point rather than a defect. The bound is what can be said knowing only the variance, and knowing only the variance turns out to be knowing very little. Every sharper number requires an assumption about shape, and the sharper the number, the more the assumption is doing.

The practical consequence is a ranking of how much trust each statement can bear. The two-sigma band is robust across shapes and worth using. The three-sigma band is a normal-only statement and should be labelled as one. And anything past four sigma is an extrapolation about a tail nobody has enough data to have seen, which is why the tail is where the approximation is worst and where it is most often relied upon.

Reading the numbers off the figure rather than remembering them #

A small practical note, since the three numbers are usually carried as memorised facts and memorised facts decay in a particular direction.

The reliable ones are 68, 95 and 99.7, and the failure mode is that the last is remembered as “99.9” or “three nines”. The difference matters more than it looks: 99.7% is one event in 370 and 99.9% is one in a thousand, so misremembering the third number understates the frequency of a three-sigma event by nearly a factor of three.

The band that is worth memorising instead is 1.96, because it is the one that appears in every interval and every test at the conventional level. Two sigma is a convenient rounding of it, and the rounding costs about half a percentage point of coverage — 95.4% against the 95.0% intended, which is conservative and harmless.

Beyond three sigma the sensible practice is to compute rather than recall. Four sigma is one in 15,787 and five is one in 1.7 million, and almost nobody carries those correctly. They are also the numbers most likely to be quoted in an argument where the distribution is not normal, which is the failure the section above is about.