Themes
Count it, do not claim it
An interval that says 95% is making a statement about a procedure, and a procedure can be run ten thousand times. Every coverage figure on this site is a count — for a proportion it is an exact sum over the whole sample space, so the answer carries no simulation noise at all.
One run is an anecdote
A figure showing a single simulation is showing one draw from a distribution of figures. Where the claim is about behaviour rather than about one dataset, the figure runs across many seeds and reports what held for all of them — and where it shows one run, it shows twenty of them side by side.
Two routes to a number
Every simulation here has a closed form beside it and every closed form has a simulation. A distribution function and its quantile must invert each other to ten digits; an exact coverage sum and a Monte Carlo count must agree. Neither route can confirm itself, and they share no arithmetic.
The tail is where it is read
A p-value, a control limit and a risk figure are all tail statements, and the tail is exactly where every approximation in this subject is worst. The central limit theorem converges in the middle long before it converges where anyone looks.
The second number
A p-value cannot be interpreted alone. The same 0.04 corresponds to a large effect at ten observations and a negligible one at two thousand, and it means nothing at all until you know how many analyses were available to produce it.
Reversals that are nobody's mistake
Simpson's paradox, regression to the mean and the winner's curse all arise from correct arithmetic applied honestly. Each is shown as a region of a parameter space rather than one famous table, so the questions of how often and how large have answers.