A unit that has not moved

A rule that admits only assignments whose covariate imbalance is within a tolerance can cut its admissible set into pieces a swap-one-pair chain cannot cross, and the randomisation test built on such a chain then reports a p-value over the piece it started in. The diagnostic at two hundred pointed the field’s two-chain diagnostic at a trial size no enumeration reaches and found it giving three answers instead of one. Its statistic divides by a standard error built from the chain’s own autocorrelation time, and at two hundred units a run of tens of thousands of steps does not contain enough independent information to estimate its own relaxation time. The essay ended by naming the repair: a longer run, or a different instrument.

A ceiling that is one unit supplied the different instrument. On fourteen units, where the set can be enumerated, it found that the split is usually a single unit — one that sits in one arm on every assignment of one component and in the other arm on every assignment of the other — and that a test reading which units moved in two chains from an assignment and its mirror needs no standard error at all. Its verdict is a count: any unit that neither chain has moved, which, by the mirror, sits in opposite arms in the two. That essay’s last section asked whether units freeze at two hundred. This one runs the test there.

The setting, and what is recorded #

The design is the one the earlier essay used at scale: two hundred units, a covariate drawn fresh for each design, a rule balancing the covariate, its square and a median cut within a tolerance, and six tolerances from 1.0 down to 0.16. Five designs are drawn. On each, at each tolerance, two chains are started — one from a random admissible assignment, one from its mirror — and run for 1.6 million steps each. For every unit, each chain records the first step at which the unit’s arm differs from where that chain began. From that one array the unit test’s verdict can be read at every run length up to the longest, so 25,000 steps, 100,000, 400,000 and 1.6 million are four readings of one run rather than four runs.

At the loosest tolerance half the units have moved within 79 steps and every unit within about a thousand. Tightening the tolerance stretches the curve at both ends, and the right-hand end most: at 0.3 nine units in ten have moved by step 645 and the last by step 3,878; at 0.22 nine in ten by step 747, while one unit in the first chain does not move until step 1,124,534. At 0.16 one unit never moves in 1.6 million steps. The curve is flat for most of its length and then rises like a wall, and the wall is a handful of units.

A count read at four lengths #

At 25,000 steps — a run of the length a practitioner would choose, and the length the earlier essay found too short for its own statistic — the unit test fires on four of the five designs at a tolerance of 0.16 and on two at 0.22. At 100,000 steps it fires on three at 0.16 and two at 0.22. By 1.6 million it fires on three at 0.16 and one at 0.22, and nowhere at 0.3 or looser. So the count does fall with the run, as a count of units the chain has not yet reached should, and on four tolerance-design pairs it stops falling.

The ones that stopped falling are the interesting ones and the ones that kept falling are the warning. The third design at 0.22 had a unit unmoved in both chains at 100,000 steps; it moved at about 109,000 in one chain and 120,000 in the other. The fifth design at 0.16 had one at 25,000 steps and none by 100,000. A test read at 100,000 steps would have called both split, and both are not — or at least not by that unit. The unit test is free of an estimated standard error and is not free of the run length: a unit the chain has not reached yet looks exactly like one the set will not let move.

A unit no swap can move #

Which of the remaining four is which can be settled without running anything longer, and it is settled by arithmetic.

Every admissible assignment has each balanced column’s scaled imbalance inside $[-a, a]$. A swap of a treated unit $i$ for a control unit $j$ changes each column’s total by twice the difference between the two units’ values in that column. If that change, scaled, exceeds $2a$ in some column, the swap cannot take one admissible assignment to another — from any assignment, because no starting imbalance inside $[-a, a]$ can absorb a jump wider than the window. A unit for which every possible partner breaks that bound in some column cannot change arm from any admissible assignment at all. It is fixed in whichever arm the chain began with, its mirror is fixed in the other, and the admissible set is split into at least two pieces that no swap-one-pair chain can cross.

The bound is a sufficient condition and not a necessary one — a swap small enough in every column can still be blocked by where the imbalance happens to sit — but it is easy to compute: for each of two hundred units, each of 199 partners, each of three columns. On the five designs and six tolerances, it freezes exactly the units the 1.6-million-step chains left in place: unit 171 on the first design at 0.16, unit 80 on the third at 0.16, unit 103 on the fourth at both 0.22 and 0.16, and nothing anywhere else. Every unit the long runs could not move is a unit nothing could move, and no unit the bound freezes escaped the runs.

So the earlier essay’s open question has an answer at these tolerances, and it is a proof rather than a diagnostic: at two hundred units and a tolerance of 0.16, the admissible set splits on three of five designs, and on one of them it already splits at 0.22.

Which unit, and why #

The frozen units have something in common, and the arithmetic says what. Unit 103 on the fourth design sits at 3.25 on the covariate, the most extreme of its two hundred; unit 80 on the third at −3.24, also the most extreme; unit 171 on the first at 2.61, the second most extreme. A unit far out on the covariate is far out on its square too, and swapping it for any other unit moves the squared column’s imbalance by twice the difference of their squares. For unit 103 the gentlest swap available is with the next most extreme unit on the same side, at 2.26, and even that moves the squared column’s scaled imbalance by 0.52 — against a window of 0.32 at a tolerance of 0.16 and 0.44 at 0.22. For unit 80 the gentlest swap moves it by 0.42, for unit 171 by 0.35, both against 0.32. No partner is close enough on the square to make the swap fit, and the partners that come closest are the units nearest it in the tail.

That is the leverage the earlier essays measured, at its limit. The unit whose sign the constraint has most to say about is the unit with the largest value on the columns the rule balances, and at a tight enough tolerance the constraint says everything: that unit stays where it was put. The prediction the earlier essay made — that frozen units would become rarer as units acquire more exchange partners, but that a tight enough tolerance on enough columns would freeze some at any size — is borne out, with a mechanism: what freezes a unit at two hundred is not a shortage of partners but an extreme value, and every sample of two hundred from a normal covariate has one.

This is also why the split was invisible to the instruments that came before. The walk that cannot cross found the admissible set in pieces by enumerating every assignment of fourteen units, which no one can do for two hundred. The two-chain diagnostic summarised each chain by a probe — a column of the covariate — and a probe is an average over two hundred units, in which one unit that never moves is a two-hundredth of the signal and one hundred and ninety-nine units that move freely are the rest. The frozen unit is the largest single fact about the set and the smallest contribution to any summary of it.

The chain is not stuck #

It is worth ruling out the obvious misreading. A chain that accepted almost nothing would leave every unit in place and the unit test would fire on everything. That is not what happens here. At the tightest tolerance the chain still accepts between 22% and 25% of its proposals on every design; it is moving freely through the part of the set it can reach, rearranging the other 199 units hundreds of thousands of times. A proposal swaps one treated unit for one control unit chosen at random, so a given unit is in about one proposal in a hundred: over 1.6 million steps, about sixteen thousand proposals would have moved unit 103, and every one was refused.

How long until every unit has moved #

The time to touch every unit is the run length the unit test needs before a quiet reading means anything, and it varies enormously. At the two loosest tolerances it is a few hundred to about two thousand steps on every design. At 0.3 it runs from about 4,000 to about 41,000 steps across the five designs; at 0.22 from about 9,000 to 1.1 million where it is finite at all, and at 0.16 from about 27,000 to 121,000 on the two designs where it is finite. A practitioner who runs 25,000 steps at 0.22 and sees every unit moved on one design has learned something; one who sees a unit still in place has learned that 25,000 steps was not enough to say.

That is a different failure from the probe test’s, and a more useful one. The probe test fails silently: it divides by a standard error it cannot estimate and reports a number. The unit test fails visibly: a unit is in place or it is not, and whether it can ever move can be checked by arithmetic before believing the chain about it.

What a practitioner should run #

The measurement turns into a procedure that needs no enumeration. First, before any chain, compute the bound: for each unit, whether any partner’s swap fits inside the tolerance window in every balanced column. Any unit that fails is frozen, the set is split, and no chain will cross it; the randomisation test’s reference distribution is over a piece, and the design should be changed — a looser tolerance, or the extreme unit handled separately. Second, if no unit is provably frozen, run two chains from an assignment and its mirror and record which units move; run until every unit has moved in both, or report the run as inconclusive. A unit that has not moved and is not provably frozen is a unit the run has not reached. The bound costs a few milliseconds for two hundred units; the chains that would otherwise be needed to suspect the same unit cost minutes, and still could not prove it.

What the procedure cannot catch is a split that is not a unit: a combination of units locked together, of the kind the ceiling essay found on five of a hundred enumerated designs at fourteen units, or a swap the bound allows that the imbalance’s position always blocks. Neither shows as a frozen unit, and the probe test remains the instrument for those — with the size sweep’s warning that its own statistic is least reliable exactly where the set is largest.

How the numbers were made #

Five designs of two hundred units, drawn from the field’s design generator at five consecutive seeds; the rule balances the covariate, its square and a median cut, each column standardised and the box tolerance applied to each scaled imbalance. At each tolerance a starting assignment is hunted for by rejection, its mirror is its complement, and each is the start of one swap-one-pair chain of 1.6 million steps with no thinning. Every step updates, for each unit not yet moved, a comparison of its current arm with the start. The bound is computed on the same columns and the same scale the chain’s acceptance rule uses, so a unit the bound freezes is one the chain’s own rule could never move.

Checked on the measurement: after 1.6 million steps the units unmoved in both chains are, unit for unit, the units the bound freezes, on every design and tolerance; at the tightest tolerance some unit never moves while at the loosest every unit moves within five thousand steps; and at the tightest tolerance every chain still accepts more than one proposal in seven. The refusal is the reading a short run invites — that a unit unmoved in both chains at a hundred thousand steps is frozen — and the third design at 0.22 breaks it.

Still open: a bound that reads where the imbalance sits #

The bound used here ignores where the imbalance is, so it freezes only units whose every swap is too big for any assignment. A sharper bound would ask whether a swap that fits the window’s width can ever fit its position — whether some admissible assignment has room in the right direction in every column at once — which is a small linear feasibility problem per unit and partner. It would catch splits the width bound misses, and the measurement that would show whether it matters is the count of units the long chains leave in place that the width bound does not freeze. Here that count is zero at every setting, so at these tolerances the width bound is the whole story; whether that holds at tighter tolerances or with more balanced columns has not been measured.

What is claimed here, and what is not #

This essay takes the unit test at two hundred units. The claims are that over 1.6 million steps a unit stays in place in both chains on three of five designs at a tolerance of 0.16 and one at 0.22, and on none at 0.3 or looser; that at shorter runs the test also fires on design-tolerance pairs where every unit eventually moves, one of them not until about 120,000 steps; that a width bound computed from the design alone freezes exactly the units the long runs leave in place; that every frozen unit is the most or second most extreme on the covariate; and that the chains accept between 22% and 25% of proposals at the tightest tolerance, so the frozen unit is local rather than the chain stalled.

What stays out, named as a decision: splits that are not a unit, which neither the bound nor the unit test sees; other rules and column sets; and the position-aware bound, deferred above.

The boundary against the diagnostic at two hundred is that it ran the probe test at this size and found it unable to settle, and this essay runs a test that settles — or proves there is nothing to settle. Against a ceiling that is one unit, which found the frozen unit at fourteen units by enumeration, this essay finds it at two hundred by a bound.