How far to project

A probe chosen from the design found that the probe a two-chain diagnostic reads is mostly the balancing rule’s own span. A rule that balances the first three powers of a covariate and a median cut has, on fourteen units, already pinned about nine tenths of the covariate’s fourth power, so a probe built from it is measuring a quantity the rule forced near zero. Projecting it off the span — one least-squares fit against columns the rule already has — tripled the separation between the admissible set’s two components, and the next essay found the projected probe missing a split on 12% of designs where the raw one missed 44%. The design’s own leverage, projected the same way, was the best probe that needed no dictionary.

Both essays treated projection as a switch: the probe as it is, or the probe with its whole in-span part removed. The second ended on the half of the separation the enumeration keeps and no projected probe reaches, and on the observation that about an eighth of the separating direction lies inside the span. If part of the signal is in the span, removing all of the span is not obviously right. This essay turns the switch into a dial.

The dial #

A probe $v$ projected a fraction $\lambda$ of the way off the span is $v - \lambda\,(v - P_\perp v)$, where $P_\perp v$ is its residual against the rule’s balanced columns. At $\lambda = 0$ it is the probe as the earlier fields used it; at $\lambda = 1$ it is the projected probe; in between, its in-span part is shrunk rather than removed.

Three probes go along the dial on the same designs: the covariate’s fourth power, standardised, which the earlier fields used as their default; the design’s leverage — each unit’s diagonal of the hat matrix of the rule’s columns, a fact about the design and the rule alone; and the separating direction itself, the difference between the two components’ centroids, which needs the enumeration and is the benchmark. Two hundred designs of fourteen units are drawn and the admissible set of each is enumerated; 100 are split, and on each of those the separation a probe carries is the difference between the components’ mean probe values over the spread of the probe inside a component. The medians below are over the designs on which a probe’s separation is finite — 82 for the fourth power and for leverage, 79 for the separating direction — which is the convention the earlier essays used.

The fourth power wants all of it #

The fourth power’s separation rises at every step of the dial: 1.543 unprojected, 2.570 half way, 3.686 three quarters of the way, 5.080 fully projected. The field’s choice was the right end of the dial for this probe, and the gain is concentrated at the end: the last quarter of the dial adds more than the first half, and no point short of nine tenths of the way comes within a quarter of the full projection’s separation.

The reason is in how much of it the rule had pinned. 92.2% of the fourth power’s variation across these designs lies in the span of the rule’s columns, so the unprojected probe is mostly a measurement of the imbalance the rule removed, which is the same in both components by construction and contributes only within-component spread. Shrinking that part by half still leaves most of the probe’s spread in it; only removing nearly all of it lets the small out-of-span signal dominate.

Leverage wants less than half #

Leverage behaves differently. Unprojected it already carries 4.537, nearly three times the raw fourth power and close to the projected one. Along the dial it rises to 4.924 at four tenths of the way and then falls, to 3.836 fully projected. The field’s leverage probe was the fully projected one; it was 22% below the best point on its own dial and below the unprojected leverage too.

So the field’s recommendation for leverage — project it as it projected the dictionary — was the wrong end of the dial. Projection is not a property of the method; it is a property of the probe. A dictionary column is mostly the rule’s span and wants all of it removed. Leverage is a function of the span — it is built from the rule’s own columns — and its in-span part is not noise the rule has pinned: it is where the design’s influential units are.

Leverage’s in-span share is 79.4%, lower than the fourth power’s but still most of it. What makes the difference is not how much of each probe is in the span but what that part does. For the fourth power the in-span part is the same in both components and only adds spread; for leverage some of it differs between components — which is what one would expect if the units whose constraint pins the set into two pieces are the high-leverage ones, since leverage is made of the span. How far, not whether found that each unit’s mean exchange depth is exactly its leverage, which is the same fact from the walk’s side.

Even the separating direction is not at the end of the dial #

The separating direction has 12.7% of itself in the span — the eighth the earlier essay named. If that part were pure signal, projecting would only lose, and fully projected the separating direction does lose: 10.565 unprojected, 10.273 projected. But a quarter of the way along it carries 11.500, more than unprojected.

That says something about the benchmark rather than about the probes. The separating direction is the difference between the two components’ centroids. It is the direction along which their means differ most, not the direction along which they are most separated relative to their spread, and those are different directions whenever the spread inside a component is not the same in every direction. Part of the centroid difference’s in-span portion lies along directions where the components spread a great deal, and shrinking it trades a little of the mean difference for a larger cut in the spread. A Fisher discriminant would build that trade in from the start; the centroid difference does not.

So the earlier essay’s ceiling — 10.565, the oracle the feasible probes were half of — was not quite the ceiling. It is a point a quarter-projection improves on by about nine per cent, and the true maximum of the separation over all directions is above both.

Three columns from outside the rule #

The fourth power is one dictionary column. Whether “a column from outside the rule wants the whole projection” is a fact about it or about the kind of probe is checked by running the same dial on two more columns the rule was not handed: a cut at one standard deviation above the covariate’s mean, and a cut at one below. On fourteen units neither exists on every design — a cut needs units on both sides of it — so their medians rest on 36 and 40 designs.

Both behave like the fourth power. The cut at one has 87.2% of itself in the span and its separation rises from 1.083 unprojected to 3.539 fully projected; the cut at minus one has 84.3% in the span and rises from 0.938 to 2.972. Every point of both dials is higher than the one before, and the best point is the far end.

What the three have in common is that none of them is built from the rule’s columns. Each happens to correlate with those columns on a small sample — a cut at one is correlated with the covariate and its square, a fourth power with the square — and the correlation is exactly the part the rule has pinned. Removing all of it costs nothing because none of it was about the split. Leverage is the opposite case: it is computed from the rule’s columns, so its in-span part is not a coincidence of the sample but the probe’s whole construction, and some of it is about the split.

What the dial does to the earlier ranking #

The essay that chose probes ended on a ranking of six, by median separation: the most concentrated direction off the span at 0.543, a random direction off the span at 0.942, the fourth power as it was used at 1.543, leverage off the span at 3.836, the fourth power off the span at 5.080, and the separating direction at 10.565.

The dial moves one place and leaves the order otherwise alone. Leverage at its best point, 4.924, is about three per cent below the projected fourth power instead of a quarter below it, and it gets there without a dictionary: a practitioner who has only the design and the rule now has a probe nearly as good as the best probe a dictionary supplied. The projected fourth power stays at the top of the feasible probes, and the separating direction stays above all of them, at a level a quarter-projection raises rather than at a fixed ceiling.

That last point matters for how the earlier essays’ “half of what the enumeration gives” should be read. The feasible probes reach about half of the separating direction’s separation; the separating direction itself is not the maximum, so they reach somewhat less than half of what a probe could carry. The gap the earlier essay named is real and slightly wider than it said.

Alignment, read along the same dial #

Separation is what the diagnostic acts on. Alignment — the correlation between a probe and the separating direction — is the geometric reading, and it moves differently.

The fourth power’s alignment rises from 0.172 to 0.658 along the dial, with its separation. Leverage’s rises from 0.426 unprojected to a peak of 0.560 at nine tenths of the way, and is 0.540 fully projected — so by alignment leverage wants nearly the whole projection, while by separation it wants less than half. The separating direction’s alignment with itself falls from 1 to 0.933 as its in-span eighth is removed.

The two readings disagree on leverage because alignment ignores the within-component spread that separation divides by. Projecting leverage further makes it point a little more along the centroid difference and spread a great deal less evenly inside each component, and the separation counts the second as well as the first. A diagnostic reads separation, because its verdict is a difference in means against the run’s own noise, so the dial should be read on separation.

The share in the span does not decide it #

A tempting shortcut would read the dial’s best point off the one quantity a practitioner can compute without the enumeration: how much of the probe lies in the span. It is a single projection, it needs no outcome and no chain, and it plainly matters — a probe that is nine tenths span is mostly measuring what the rule pinned.

It does not decide the dial. The five probes’ in-span shares and best points are 92.2% and full projection for the fourth power, 87.2% and full for the cut at one, 84.3% and full for the cut at minus one, 79.4% and four tenths for leverage, and 12.7% and a quarter for the separating direction. Leverage sits between the dictionary columns on the share and nowhere near them on the dial. A rule of the form “project more when more of the probe is in the span” would put leverage at nearly full projection, which is the field’s original choice and the one this essay finds a fifth below its best.

What separates leverage from the cut at minus one, whose shares differ by five points, is where the in-span part came from. A basis is a subspace, and a function of the design that is defined through that subspace — the hat matrix’s diagonal — carries the subspace’s structure, including the structure that decides which units the walk cannot carry across. A dictionary column that merely correlates with the subspace on fourteen units carries the correlation and nothing else. The share is the same kind of number for both; the content of the shared part is not, and only the enumeration — or an argument about how the probe was built — can tell them apart.

What a practitioner can choose without the enumeration #

The dial’s best point was found on enumerated designs, and a trial does not have the enumeration. What it does have is the two facts that decided where each probe’s best point sat.

A probe built from outside the rule’s columns — a dictionary function the rule was not handed — is mostly the rule’s span on a small trial and should be projected all the way. That is the earlier essays’ recommendation and it stands for every dictionary column, with nothing to tune.

A probe built from the rule’s own columns — leverage, or anything else that is a function of the design and the basis — is about the span rather than merely in it, and should not be projected fully. On these designs a projection of between a quarter and a half of the way kept it within two per cent of its best, and the unprojected leverage was within eight per cent. A fixed λ of four tenths, chosen once from this table, is the simplest rule that uses the finding, and it asks nothing of the trial beyond the one projection the earlier essays already required.

That second recommendation rests on one setting — fourteen units, one basis, one tolerance — and the dial’s best point is a measured fact about that setting, not a constant. What transports is the distinction between a probe that is in the span by accident and one that is in it by construction.

Still open: a direction built for the ratio #

The separating direction’s own dial says the benchmark the feasible probes were measured against maximises the wrong thing. The direction that maximises separation — the components’ mean difference whitened by their within-component spread — is computable on enumerated designs and is the true ceiling for any probe. How far above 11.5 it sits, and how much of the remaining gap between the best feasible probe and it is in the span, is the measurement this essay points to. If most of it is, the half of the separation no projected probe reaches is reachable by a probe that keeps the right part of the span; if little of it is, the enumeration’s advantage is in directions no function of the design can find.

The claim, and where it stops #

This essay takes projection off the rule’s span as a dial rather than a switch. The claims are that over a hundred enumerated split designs the fourth power’s median separation rises from 1.543 unprojected to 5.080 fully projected and is highest at full projection; that leverage carries 4.537 unprojected, peaks at 4.924 four tenths of the way and falls to 3.836 fully projected; that the separating direction loses separation to full projection, 10.565 to 10.273, and gains it to a quarter-projection, 11.500; that the three probes have 92.2%, 79.4% and 12.7% of themselves in the span; and that by alignment leverage peaks near full projection while by separation it peaks before half way.

What stays out, named as a decision: other designs, bases and tolerances, on which the diagnostic’s two moments would also have to be re-read; the chain’s verdict at each point on the dial, which the detection essay measured only at the two ends and which would take a two-chain test per design per λ; and the Fisher direction, deferred above. The boundary against the part the rule already took is that it measured how much of a probe the rule pins and this essay how much of that to give back.