Normal Distribution
nothing is called 95% until it has been counted
Concept

False-alarm — where it appears

A diagnostic or test that declares a defect where there is none, counted as a rate over cases known to be clean. It is the other half of a detection rate: an instrument that fires on everything finds every real defect and says nothing.

Named by 3 essays across 3 fields — each of them below, with the objects they name alongside it.

One unit that never changes arm. Every admissible assignment of one design of fourteen units — 192 of them, enumerated, in two components of 96 and 96 — read unit by unit: for each unit, the share of a component's assignments that put it in the treated arm, in the first component (filled) and the second (open). Most units are treated on between a third and two thirds of either component's assignments. One unit, the one at x = 0.28, is treated on 0% of the first component's assignments and 100% of the second's. Asking which arm that unit is in tells the two components apart without error, which no linear probe built from the covariate does.

A ceiling that is one unit

The benchmark every probe of a split admissible set was measured against maximised the wrong thing, so the obvious next step was the direction that maximises the separation itself. On 88 of 100 split designs that direction has no finite value: some combination of units reads the same on every assignment of a component. On 83 of them it is a single unit, sitting in one arm across one component and the other arm across the other — and a chain can see a unit that never moves without enumerating anything.

aimed · Randomisation When each unit first changed arm. One design of two hundred units, the rule balancing the covariate, its square and a median cut, and a swap-one-pair chain run for 1.6 million steps from a random admissible start, at six tolerances: for each unit, the step at which its arm first differed from the start, the two hundred sorted from first to last. At 1: half the units had moved by step 79, nine in ten by 269, all by 1,073; At 0.6: half the units had moved by step 79, nine in ten by 362, all by 1,073; At 0.4: half the units had moved by step 106, nine in ten by 429, all by 1,593; At 0.3: half the units had moved by step 153, nine in ten by 645, all by 3,878; At 0.22: half the units had moved by step 194, nine in ten by 747, all by 1.1 million; At 0.16: half the units had moved by step 237, nine in ten by 1270, and 1 never moved. The last units to move set the run length a diagnostic built on them needs, and at the tightest tolerance one never does.

A unit that has not moved

At two hundred units the two-chain diagnostic could not settle its own verdict, because its standard error needed a relaxation time the run was too short to estimate. Reading which units moved needs no standard error. Run for 1.6 million steps on five designs, it finds a unit in place in both chains on three designs at the tightest tolerance and one at the next — and every one of them is a unit no swap can move from any admissible assignment, by arithmetic. The admissible set does split at two hundred units, and the split is one extreme unit.

reach · Randomisation How many starts it takes to land in two pieces. Averaged over the split designs at each of three tolerances, the chance that k starting assignments drawn uniformly from the admissible set — what hunting for a start by rejection gives — do not all fall in one component, which is the most any diagnostic built on comparing chains from those starts can detect. At 0.6: 28% with 2, 47% with 4, 63% with 8, 77% with 16, 88% with 32, 95% with 64, 99% with 128. At 0.8: 12% with 2, 21% with 4, 32% with 8, 46% with 16, 62% with 32, 77% with 64, 90% with 128. At 1.0: 4% with 2, 8% with 4, 13% with 8, 21% with 16, 33% with 32, 47% with 64, 64% with 128. The median split design needs 21, 109 and 439 starts at the three tolerances to expose its split with probability 95%. With an even allocation the mirror makes two starts enough whenever the split is a unit.

After the mirror is gone

The two-chain diagnostic starts its second chain at the complement of the first's start, which exists only when the arms are equal. At five treated against ten, enumerated on two hundred designs, a split admissible set is one big piece and some crumbs — the median puts 97.3% of its assignments in one component — so two random starts see the split on 12% of split designs and 95% needs a median of 109 starts. Worse, the rule fixes units in a connected set all the time, so a unit that never moves stops being evidence. A hunt that forces such a unit into the other arm finds 41 of 70 splits; 29 carry no unit that marks them.

after · Randomisation

Named alongside it

The objects these essays reach for when they reach for this one.

Covariate balanceMarkov chainRandomisation testAdmissible setEnumerationLeverageAssignment mechanismConnected componentConnectivityExact enumerationImbalanceMixing

All concepts