Normal Distribution
nothing is called 95% until it has been counted
Field

A probe chosen rather than picked

A diagnostic that reports on what a balancing rule was not handed is run through a column that is 92% inside the span the rule balanced — because orthogonality in the population is not orthogonality on fourteen units. Projecting the probe off that span costs one least-squares fit and triples the separation it carries; choosing the direction from the design's own leverage is worth another factor of four over a random one; and the projection-pursuit direction the argument invites is worse than random. On a short chain the raw probe misses 44% of the sets that are split and the projected one misses 12%.
What is left of a probe after the rule has had it. The share of each dictionary function a rule balancing x, x2, x3, cut0 has already taken, on trials of 14 units, averaged over 100 designs. Four of the eight functions are the basis, so their share is exactly one: a randomisation test run on one of them is asking about a quantity the rule forced to zero, and one of them is the default probe of the field this measurement comes from. The four that are not still read 0.919, 0.873, 0.903, 0.832 — between 0.832 and 0.919 of them is inside the span — against closed-form removed shares of 0.000, 0.692, 0.590, 0.692. At 14 units a rule with four functions in it takes most of anything it is shown.

The part the rule already took

A diagnostic that reports on what a balancing rule was not handed is run through a column that is 92% inside the span the rule balanced — because orthogonality in the population is not orthogonality on fourteen units.

 How far apart the two components are, on each probe. The median separation between the two components of the admissible set — the difference in their mean probe values, over the spread inside a component — over the 100 of 200 designs whose set is enumerated and found split. The separating direction carries 10.565 and needs the enumeration. The fourth power as the earlier fields use it carries 1.543; projected off the span the rule balances, 5.080. The design's own leverage, which uses no dictionary and no outcome, carries 3.836. A random direction in the same subspace carries 0.942, and a direction chosen by looking for concentrated structure carries 0.543 — below random, and the one heuristic here that is worse than not choosing at all.

A probe chosen from the design

The design's own leverage aligns with the separating direction four times better than a random direction in the same subspace. The concentrated direction the argument invites is worse than random.

 How often each probe finds a split that is there. The share of 34 designs — every one of them enumerated to be in two components — on which a two-chain test of 800 draws declares the split, by probe. The fourth power as the earlier fields use it finds it on 55.9%, so it misses 44.1% of the sets that have one. The same column projected off the rule's span finds it on 88.2%, and the separating direction itself on 91.2%. The design's own leverage, chosen without any dictionary, gets 79.4%. A random direction in the same subspace gets 44.1%, and the direction chosen for being concentrated gets 38.2% — worse than random, which is what a heuristic that finds the wrong structure looks like from the outside.

What a chosen probe finds

On a chain of eight hundred draws the probe the earlier fields use misses 44% of the sets that are split. Its own residual off the rule's span misses 12%, for one least-squares fit.

 Projection as a dial. Each probe projected a fraction λ of the way off the span of the rule's balanced columns, over the 100 designs of 200 whose admissible set is enumerated to be split; the median separation — the two components' difference in mean probe value over the spread inside a component. the separating direction: 10.565 at 0, 11.500 at 0.25, 11.456 at 0.4, 11.424 at 0.5, 11.240 at 0.6, 10.664 at 0.75, 10.427 at 0.9, 10.273 at 1; the design's leverage: 4.537 at 0, 4.845 at 0.25, 4.924 at 0.4, 4.849 at 0.5, 4.678 at 0.6, 4.417 at 0.75, 4.205 at 0.9, 3.836 at 1; the fourth power: 1.543 at 0, 1.846 at 0.25, 2.094 at 0.4, 2.570 at 0.5, 2.870 at 0.6, 3.686 at 0.75, 4.579 at 0.9, 5.080 at 1.

How far to project

The field's repair for a probe is to project it off the span the balancing rule already pinned. Treated as a dial rather than a switch, that repair is right all the way for the fourth power — its separation rises from 1.543 unprojected to 5.080 fully projected — and wrong for the design's own leverage, which carries 4.537 unprojected, 4.924 at four tenths of the way and only 3.836 projected fully. The leverage probe the field recommended was a third of the way past its best. Even the separating direction itself gains a little from a partial projection and loses a little from a full one.

 One unit that never changes arm. Every admissible assignment of one design of fourteen units — 192 of them, enumerated, in two components of 96 and 96 — read unit by unit: for each unit, the share of a component's assignments that put it in the treated arm, in the first component (filled) and the second (open). Most units are treated on between a third and two thirds of either component's assignments. One unit, the one at x = 0.28, is treated on 0% of the first component's assignments and 100% of the second's. Asking which arm that unit is in tells the two components apart without error, which no linear probe built from the covariate does.

A ceiling that is one unit

The benchmark every probe of a split admissible set was measured against maximised the wrong thing, so the obvious next step was the direction that maximises the separation itself. On 88 of 100 split designs that direction has no finite value: some combination of units reads the same on every assignment of a component. On 83 of them it is a single unit, sitting in one arm across one component and the other arm across the other — and a chain can see a unit that never moves without enumerating anything.

All essays