Normal Distribution
nothing is called 95% until it has been counted
Concept

Expected loss — where it appears

The average cost of a decision rule, weighting each outcome's cost by its probability. Comparing rules by expected loss turns a choice of threshold or level from a convention into a calculation once the costs of each kind of error are stated.

Named by 3 essays across 2 fields — each of them below, with the objects they name alongside it.

The operating characteristic, and the point that minimises harm at 0.10% prevalence. The published pair — 90% sensitive, 95% specific — is the open mark. With a miss costing 100 times a false alarm and a prevalence of 0.10%, the threshold that minimises expected cost sits at 74.7% sensitivity and 98.81% specificity, with a predictive value of 5.9%.

The test is a point somebody chose

A test reported as 90% sensitive and 95% specific is not two properties of a test. It is one property read at a threshold, and the threshold that minimises harm runs from 3.05 standard deviations of the score at a prevalence of one in ten thousand to −0.12 at one in two — 45% of cases detected at one end and 99.9% at the other.

screening · Baserate The share of healthy people the harm-minimising threshold flags, against the prevalence, for three spreads of the diseased scores. A miss costs a hundred false alarms. With diseased scores as spread as healthy ones the best threshold flags more healthy people smoothly as the prevalence rises. With them twice as spread it flags 81.1% just below a prevalence of 25.7% and everyone just above it; three times as spread, 45.7% and then everyone at 13.0%.

A threshold that jumps

When a test's scores are equally spread in the healthy and the diseased, the harm-minimising threshold slides smoothly as the prevalence changes. Make the diseased scores twice as spread — the ordinary shape of a group that mixes mild and severe cases — and keep the published 90/95 pair, and the best threshold flags 81.1% of healthy people at a prevalence just under 25.7% and every healthy person just above it. At three times the spread it jumps from 45.7% to everyone at 13.0%. A threshold that jumps cannot be given as a formula; it has to be given with the second-best point beside it.

screening · Baserate Expected loss of three upper limits against the level each is set at, fifteen exponential observations, penalty 20. A failure costs 20 standard deviations of margin. At the conventional 97.5% the t limit's loss is 2.143, Hall's 1.922 and the fitted family's 1.530. Each at its own best level: the t limit 1.263 at 99.95%, Hall's 1.862 at 98.50%, the family 1.308 at 99.25%. The best fixed multiple of the standard error costs 1.255.

The level a limit should be set at

When a failure costs twenty times a standard deviation of margin, a t limit on fifteen exponential observations set at the conventional 97.5% has an expected loss of 2.143. Set at 99.95%, the same formula's loss is 1.263 — within 1% of the best any fixed multiple of s/√n can do, and ahead of both Hall's transformation and the fitted gamma family at their own best levels. Hall's is the worst at every level on every source. Choosing the level does more than choosing the construction, and the correction that reads the sample is the one no level rescues.

tails · Student

Named alongside it

The objects these essays reach for when they reach for this one.

Base rateDecision thresholdROC curveScreeningSensitivity and specificityCoverageDecision theoryLikelihood ratioPredictive valuePrevalenceSample sizeSignificance level

All concepts