Tests, and the second number

A p-value that is not flat is not a p-value

Under a true null, p-values are uniform. That is stronger than saying the test rejects 5% of the time, it constrains the whole distribution rather than one point of it, and it catches implementation errors that a rejection rate sails past.

A p-value is the probability, if the null hypothesis is true, of a statistic at least as extreme as the one observed. Read that definition carefully and a consequence falls out immediately: under the null, a p-value is the value of its own distribution function, and is therefore uniform on (0, 1).

20,000 p-values from a true null, n = 12Flat, as it must be: under the null a p-value is uniform on (0,1). The Kolmogorov–Smirnov distance from uniform is 0.0165 (p = 0.13). That flatness is the check that catches an error a single rejection rate would miss.05001e+30.2000.4000.6000.800p-valuecountflatKS distance 0.0165a p-value that is not uniform is not a p-value
Fig. 1 Twenty thousand p-values from a test where the null is true by construction. Flat, as it has to be.
What a one-sample t test at n = 20 can detectAt an effect of 0.5 standard deviations the test finds it 56% of the time. Below that, a non-significant result is the expected outcome of a real effect — which is why "no significant difference" is not evidence of no difference.00.2500.5000.750100.5001true effect, in standard deviationsprobability of rejecting80%, the usual target56% at d = 0.54,000 simulated studies per pointthe second number a p-value needs
Fig. 2 The same measurement taken under an alternative rather than the null. Flatness is the baseline this is measured against.

Why flatness is the better check

The obvious check on a test is its rejection rate: run it under the null and confirm that about 5% of runs give p < 0.05. That is worth doing and it is weak.

A rejection rate checks the distribution at one point. A test can have exactly the right rate at 0.05 and be wrong everywhere else — too many small p-values compensated by too few moderate ones, a distribution that is right in the far tail and wrong in the body, or a systematic distortion that happens to cross the correct value at the threshold being checked.

Uniformity checks the whole distribution. Every quantile has to be right simultaneously, which is a far more demanding condition and correspondingly harder to satisfy by accident.

The practical difference is that flatness catches the errors people actually make:

  • a wrong degrees of freedom, which shifts the whole distribution
  • a one-sided p reported as two-sided, or the reverse
  • a variance computed with n instead of n − 1
  • a test statistic missing a scaling factor

Each of those breaks flatness immediately. Several of them can leave the rejection rate at 0.05 looking roughly acceptable.

The false-positive rate against the number of analyses, on pure noiseThe data has no effect in it. Each analysis is correct and each p-value is honest. With 20 correlated outcomes available, something reaches p below 0.05 57% of the time.00.2000.4000.6000.8005101520analyses available to the researcherchance of finding something significantthe nominal 5%if the analyses were independentcorrelated, as real ones areno effect present anywhereevery individual analysis is correct
Fig. 3 What happens to the distribution when several analyses are available and the smallest p-value is reported.

The check, checked

An assertion that has never been shown to reject anything proves nothing, so the site’s gate feeds this check something it must refuse.

It takes the same t test and halves every p-value — the classic one-sided/two-sided slip, and a mistake that produces plausible-looking output. The rejection rate roughly doubles, which a careful reviewer might notice. The uniformity check does not need care: the distribution is crushed into the lower half of the range and the Kolmogorov–Smirnov statistic detects it overwhelmingly.

The gate requires that refusal to happen. If halving the p-values stopped being detected, the check would have quietly stopped working and the build fails.

12,000 studies of a real effect of 0.3, n = 16Power is 20%. The studies that reached significance report a mean effect of 0.613 — 2.04 times the truth. Every one of them is honest; the selection did the inflating.02505007501e+3-0.50000.5001effect the study reportsstudiesthe truth, 0.3what gets published, 0.61shaded: reached p below 0.05power 20%, inflation 2.04×
Fig. 4 Conditioning on significance, which is what turns a flat distribution into a biased literature.

Reading the flatness correctly

Two things the figure does not say.

A flat histogram does not mean the test is right for a given dataset. It means the implementation is consistent with its own assumptions. Here the data is genuinely normal and independent, which is exactly what the t test assumes. Feed the same correct implementation correlated data and the p-values stop being uniform — which is useful, and is a check on the assumptions rather than the code.

The flatness is a property of the null, not of the world. Under an alternative, p-values pile up near zero, and how fast they pile up is the power. The uniform case is the baseline against which that is measured, which is why the power curve is the same measurement taken at a different truth.

The effect behind a p-value of 0.04, at three sample sizesA p of 0.04 at ten observations needs an effect of 0.76 standard deviations; at two thousand it needs 0.046. The p-value alone does not say which of these happened, which is why it should never be reported alone.ten observations0.758 sdp = 0.04a hundred0.208 sdp = 0.04two thousand0.046 sdp = 0.04the effect that gives the same p-valueall three reach p = 0.04the number does not say how big the effect is
Fig. 5 The effect behind a fixed p-value at three sample sizes. Uniformity is why the number alone carries so little.

What this implies about a single p-value

If p-values are uniform under the null, then under the null every value is as likely as every other. A p of 0.04 and a p of 0.96 are equally probable when nothing is going on.

That has a consequence worth sitting with: a single p-value just below a threshold is weak evidence. The likelihood ratio between “there is an effect” and “there is not” at p = 0.04 is modest under most reasonable alternatives — far more modest than the word “significant” suggests.

It also explains why a distribution of p-values across many studies is so much more informative than any one of them. A field where the null is mostly true should produce a flat distribution of published p-values; a lump just below 0.05 is a signature of selection, and the size of the lump is measurable. That is not a hypothetical technique — it is how the replication literature diagnosed the problem it went on to document.

Coverage of four nominal 95% intervals, n = 30Computed exactly by summing over all 31 possible counts, not simulated. The Wald interval drops to 26.0% and is jagged everywhere; Clopper–Pearson never falls below 95% and pays for it in width.0.4000.6000.80010.2000.4000.6000.800the true proportionactual coverage of a nominal 95% intervalnominal 95%Wald — the textbook oneWilsonAgresti–CoullClopper–Pearsonsummed over all 31 outcomesno interval is 95% until it is counted
Fig. 6 The same principle for intervals: check the whole range rather than one point of it.

The habit

The transferable point is not about p-values. It is that the strongest checks constrain a whole distribution rather than a single summary.

The same idea appears everywhere on this site. An interval is checked by its coverage across the whole range of the parameter rather than at one value. A random generator is checked in two dimensions rather than one, because a bad one passes the one-dimensional test. A distribution function is checked against its own quantile at a dozen probabilities rather than at the median.

In each case the cheap check passes things the demanding check catches, and the demanding check costs almost nothing extra.

What flatness looks like when it fails

The check is only useful if the failures are recognisable, and the four common ones have distinct signatures.

A one-sided p reported as two-sided, or the reverse, compresses the distribution into half the interval. The histogram is flat over (0, 0.5) and empty above, or the mirror image. Unmistakable once seen and easy to miss if only the rejection rate is checked.

A wrong degrees of freedom tilts the distribution: too many small p-values if the degrees of freedom are overstated, too few if understated. The tilt is gentle and the rejection rate can still look plausible.

A variance computed with n rather than n − 1 inflates the statistic slightly, which piles up mass at the low end. At large n it is invisible; at small n it is obvious, which is the opposite of most errors.

A discrete test statistic produces a genuinely lumpy distribution, and this one is not an error — it is the same discreteness that makes interval coverage oscillate. A test on counts cannot have uniform p-values, and expecting it to is the mistake.

That last case is why the check reports a KS distance and a p-value rather than a verdict. It is a measurement, and interpreting it requires knowing whether the statistic was continuous.

Conservative and anti-conservative

The two directions of failure have very different consequences and deserve their own names.

A test whose p-values pile up above uniform is conservative: it rejects less often than its stated level, so its false positives are under control and it has less power than advertised. The costs are invisible — missed effects and inflated estimates among those it does detect.

A test whose p-values pile up below uniform is anti-conservative: it rejects more often than stated. Its false positives exceed the advertised rate, and every claim made with it is weaker than it appears.

The second is the dangerous one and the harder to notice, because a method that finds more results looks better by every casual measure. Flatness is the check that distinguishes them and it does so in one picture.

Why this is the site’s first check

The gate runs this before anything else, and the ordering is deliberate.

Every p-value on this site comes from the same t-test implementation. Every rejection rate, every power figure, the forking-paths measurement and the winner’s curse are counts of how often that implementation crosses a threshold.

If the implementation were wrong, all of those numbers would be wrong together, in a correlated way, and each would look individually plausible. Nothing downstream would catch it, because nothing downstream has an independent route to the same answer.

Uniformity is that independent route. It constrains the implementation using a fact — p-values are uniform under the null — that follows from the definition and not from the code. That is why it goes first.

Beyond testing

The pattern transfers, and naming it is more useful than the instance.

Where a quantity has a known distribution under a stated condition, checking the whole distribution is far stronger than checking one summary of it — and usually costs the same.

A residual should be normal if the model is right: check the whole set, not the mean. A random generator should be uniform: check every bin, and in two dimensions, not just the overall average. An interval procedure should cover at 95%: check across the whole parameter range, not at one convenient value.

In every case the cheap version passes things the demanding version catches, and the demanding version is a few more lines.

The check on real data

The uniformity check above uses simulated data where the null is true by construction. On real data the null is not known to be true, so the check changes character — and becomes useful in a different way.

If a set of p-values from a real study is far from uniform, one of two things holds: there are real effects, or the test’s assumptions are violated. Those are not distinguishable from the distribution alone.

But the shape carries information. Real effects produce a pile-up near zero with a roughly flat remainder. Violated assumptions — dependence, most often — produce distortions across the whole range, including too many values near one.

So the plot is worth making for a real analysis with many tests, and worth reading as a diagnostic rather than a verdict. A distribution that is flat except for a spike at zero is the picture of a well-behaved analysis with some real findings in it.

What a p-value would need to be uniform for

The condition is more specific than it is usually stated, and the qualifications are where the practical failures live.

Uniformity requires the test statistic to be continuous and its distribution under the null to be known exactly. Both fail routinely.

A statistic on counts is discrete, so its p-values take finitely many values and cannot be uniform. They are conservative — the distribution sits above uniform — which is why exact tests on small counts have less power than their nominal level suggests. That is the same discreteness that makes interval coverage oscillate.

A statistic whose null distribution is approximated rather than known — most large-sample tests — has p-values that are only approximately uniform, and the approximation degrades in the same place every approximation on this site degrades: the tail, which is where the small p-values are.

So a plot that is flat in the body and wrong below 0.01 is the expected picture for an approximate test, and it is exactly the region that matters. Checking uniformity across the whole range, rather than eyeballing the histogram, is what catches it.

The permutation alternative

Where the null distribution is not known, it can often be constructed, and the construction gives uniformity by design.

A permutation test computes the statistic under every relabelling of the data consistent with the null. The resulting distribution is the null distribution — not an approximation to it — so the p-value is exact up to the discreteness of the permutation set.

It costs computation and it buys the property this essay is about: the p-value means what it says, without relying on a distributional assumption that a small sample cannot check.

That makes it the natural default for small samples, where the assumptions are least checkable and the approximations least reliable. It is not a niche technique; it is the honest answer to the question of how the null distribution is known, and the answer is that it is constructed.