About

This is a growing collection of illustrated essays about probability and statistical inference. Each takes a single claim and draws it until the argument is visible — and every advertised property in every figure is counted rather than quoted, with the claim about it given a test it could fail.

Why counting rather than quoting

A 95% confidence interval is making a checkable statement: build intervals this way, and 95% of them will contain the parameter. That is a claim about a procedure, and a procedure can be run. For a proportion it does not even need running — the sample space is finite, so the coverage is a sum over every outcome that could have occurred, and the answer is exact.

Doing that turns up things the textbooks do not mention. The interval taught first covers 87.6% of the time when it says 95%, and it fails worst exactly where proportions are usually reported. Coverage does not improve monotonically with sample size: nineteen observations cover 93.8% and twenty cover 81.9%. Neither of those is an estimate with error bars on it. They are sums.

So the rule this site runs on is: nothing is called 95% until its coverage has been counted. It is a low bar, it takes a few lines of code, and it is almost never done.

What is different about a subject made of samples

Every other site in this fleet draws figures from a deterministic rule, so a figure either is or is not what it claims. Here the figures are samples, and a sample can be right by luck. That changes what a figure has to carry, in two ways.

The seed is part of the figure. Nothing here calls the system random number generator. Everything comes from a counter-based generator with an explicit seed, so a figure is byte-identical on every build and anyone can reproduce it. Without that, a rebuild would change every page and no assertion could be trusted twice.

One run is an anecdote. A figure showing a single simulation is showing one draw from a distribution of figures. Where the claim is about behaviour rather than about one dataset, the assertion runs across many seeds and reports what held for all of them. Where a single run is genuinely the point, the figure shows twenty of them side by side — because the useful question is never “what happened here” but “what does this procedure do”.

Two routes to every number

A site about probability that only ever simulates has one route to each number and no way to tell a right answer from a plausible one. A site that only ever quotes formulae never finds out whether it implemented them correctly. So everything important here is available both ways and the two are required to agree.

Every distribution function and its quantile invert each other to better than ten digits, computed by unrelated arithmetic — a continued fraction one way, a rational approximation refined by Halley the other. An exact coverage sum and a Monte Carlo count of the same procedure agree to within the simulation's own standard error. And under a true null, a test's p-values must be uniform: a check that constrains the whole distribution rather than one point of it, and that catches a wrong degrees of freedom or a one-sided slip that no single rejection rate would reveal.

What the site does not do

It explains what a method computes. It does not tell anyone what to conclude from their data, and it is not a substitute for a statistician who knows the study.

That distinction carries more weight in this subject than in most, because statistical claims get applied to medicine and to policy, and the gap between what a procedure computes and what people conclude from it is where the damage happens. Several essays here are about exactly that gap — what a p-value does not say, what a positive test is worth when the condition is rare, why an underpowered significant result overstates the effect it found. Those are arguments about interpretation supported by measurements, and the measurements are the part this site can stand behind.

Everything is generated. There are no stock images, no traced diagrams and no figures fitted to make a point. Where a claim on this site turned out to be false when measured, the measurement is what got published.