What the ellipse spends its assignments on

When the constraints run out counted, exactly, what a basis does to the randomisation it leaves. Sixteen units, all 12,870 equal splits, a tolerance on each function’s standardised imbalance: 3,874 admissible assignments with one function balanced, 1,006 with two, 314 with three, and none at all with four. The experimenter’s own stated requirement became unsatisfiable, and a rerandomisation procedure written the obvious way would loop for ever.

That essay named the standard alternative and set it aside on purpose: a Mahalanobis criterion instead of a coordinate-wise one, which is the standard form of rerandomisation and whose acceptance probability is chosen rather than implied. It was set aside because the coordinate-wise box made the count a consequence of the tolerance rather than a parameter. This essay takes the alternative up, on the same units, the same basis and the same enumeration, and asks three things the box could not be asked: whether it runs out, what it costs at the same count, and where its admissible assignments go.

A region that cannot be empty #

The box accepts an assignment when every one of the k standardised imbalances is within a tolerance of zero. The Mahalanobis rule accepts it when the vector of imbalances is inside an ellipse drawn by their own covariance: $z'\Sigma^{-1}z \le t$, with Σ the covariance of the k imbalances over every equal split and t the point of a $\chi^2$ on k degrees of freedom that leaves a stated share inside. A practitioner states the share; here it is 2.5%, which on 12,870 splits promises about 322 admissible assignments.

The enumeration gives 314, 230, 34, 86, 122 and 124 admissible assignments at one to six functions. None of them is zero, and none can be: an ellipse centred at the origin contains whichever assignment is closest to the origin in its own metric, so a rule specified by a share admits at least that one whatever the basis. The box has no such floor. Its corners are fixed in the coordinates the functions were written in, and when the functions are correlated — the third Hermite function and the covariate itself have a sample correlation of −0.703 on these sixteen units — the assignments that would satisfy every coordinate at once are simply not among the 12,870.

The counts also say something unwelcome about the share. At three functions the $\chi^2$ promised about 322 and the enumeration finds 34, a ninth of it. At sixteen units the imbalance vector is not normal, and the region a $\chi^2$ quantile draws is not the region holding the stated share. A procedure that sets its share expecting a reference distribution of three hundred points can get thirty-four, with a finest attainable p-value ten times coarser than planned. The Mahalanobis rule cannot run out; it can come close.

Why the count dips at three and recovers #

The Mahalanobis counts do not fall steadily. They go 314, 230, 34, then back up to 86, 122 and 124, and the reason is a race between two numbers that both grow with the basis.

One is the $\chi^2$ threshold itself. At a share of 2.5% the point of a $\chi^2$ on k degrees of freedom is 0.0010 at one function, 0.2158 at three and 1.2373 at six: the ellipse a stated share draws gets wider in Mahalanobis units as dimensions are added, because more dimensions put more of a normal vector’s length away from the centre.

The other is how close the nearest assignment can get. Sixteen units split eight against eight is a finite set, and its closest point to perfect balance in all k functions at once moves outward as k grows: its squared Mahalanobis length is essentially zero at one function, 0.1209 at three and 0.5113 at six. At three functions the threshold has grown less than the nearest assignment has receded, and only 34 assignments sit between them. From four on the threshold outruns it and the count recovers.

The box has the second number and not the first. Its region does not widen with k — every function gets the same tolerance — so once the nearest assignment has receded past the box’s corners there is nothing left, and nothing a stated share can do about it. That is the whole of the difference between a region that can be empty and one that cannot.

Thirty-four assignments, as a test #

A count is a reference distribution. At three functions and the stated share the ellipse admits 34 assignments, so the finest p-value a randomisation test over that set can report is 1 in 34, 0.0294. A two-sided test at 5% can then reject only when the observed statistic is the single most extreme of the 34 — a test whose real level is set by the granularity of its reference set rather than by anything the experimenter chose. At four functions, 86 assignments give a floor of 0.0116, and at six, 124 give 0.0081.

None of that is hidden from a protocol that counts. All of it is hidden from one that states a share and trusts the $\chi^2$ to deliver about 322, which at three functions it misses by a factor of nine. The resolution of the eventual test is the admissible count, and on a trial small enough for rerandomisation to bind it should be computed, not assumed.

The same count, held less tightly #

The fair comparison of two acceptance regions is at the same number of admissible assignments, so that they differ only in shape. For each basis size at which the box admits anything, the Mahalanobis rule is given exactly the box’s count — the assignments with the smallest Mahalanobis distance — and each is scored on how much imbalance it leaves in the functions the basis names.

At one function the two regions are the same interval and leave the same 0.0523 of a coin’s imbalance variance. At two functions they barely differ: the box’s worst function 0.0557, the ellipse’s 0.0579. At three functions and 314 assignments they separate. The box leaves 0.0771 in its worst function and 0.0633 on average. The ellipse leaves 0.1545 in its worst and 0.0927 on average. Admitting the same number of assignments, the ellipse holds the worst named function half as tightly.

At the tighter tolerance of 0.25 the box admits 34 assignments at three functions and the gap is the same shape: the box’s worst function 0.0389, the ellipse’s 0.0796, again a factor of two.

That is a measurement of shape alone. Both regions hold 314 assignments out of the same 12,870; both are centred on perfect balance; both are symmetric under swapping the two arms. They differ only in which 314, and the box’s choice holds every named function more tightly than the ellipse’s on its worst one.

What neither region was told about is held about equally by both. Across the three functions outside the basis — the three cut points of the dictionary — the box at three functions leaves 0.5750 of a coin’s imbalance variance and the ellipse 0.5820. The shape of the region decides how the named functions are treated, not the unnamed ones. That is what the field’s projection result predicts: what a rule does to a function outside its basis depends on the part of that function the basis can see, and both regions see the same span. Inside the span they choose different directions to tighten; outside it there is nothing for either to choose.

Where the ellipse spends #

The reason is visible once the imbalances are written in their own principal directions rather than in the functions’.

The three functions’ imbalances have a covariance whose principal directions carry 0.219, 1.005 and 1.776 times the variance of a single function’s imbalance. The smallest is a combination of the covariate and the third Hermite function — the two that are negatively correlated on these units — in which they nearly cancel. It barely varies, and almost nothing about the named functions depends on it.

The box spends nothing there. It leaves 0.490 of that direction’s own variance, and holds the other two at 0.047 and 0.020 — tight exactly in the directions along which the named functions move. The ellipse leaves 0.174, 0.060 and 0.101. It has tightened the direction nobody asked about by a factor of nearly three relative to the box, and loosened the largest direction by a factor of five to pay for it.

That is not a defect of this ellipse; it is what the Mahalanobis metric is. Writing the region in whitened coordinates, where every direction has unit variance, makes every linear combination of the basis equally important, including the ones with almost no spread. The essay that counted at four hundred units found the same property from the other side: the ellipse’s admissible set does not change when the basis is rotated within its span, where the box’s changes a third of its members. Invariance to the basis and indifference to the named functions are one property. A rule that treats every combination of the span alike cannot also favour the functions somebody wrote down.

What the normal theory promises, and what sixteen units deliver #

The case for the Mahalanobis rule rests on a clean result. If the imbalance vector were exactly normal with covariance Σ, accepting it inside $z'\Sigma^{-1}z \le t$ would leave covariance v·Σ, with

$$v \;=\; \frac{P(\chi^2_{k+2} \le t)}{P(\chi^2_k \le t)},$$

because the region is a sphere in whitened coordinates and the expected squared length inside a sphere is that ratio times k. Every direction, and therefore every named function, would keep the same share v of a coin’s imbalance variance. Equal percentage reduction in every function is the property the rule is chosen for.

At one and two functions the theory is right: it promises 0.0003 and 0.0126 and the enumeration gives 0.0003 and 0.0129 on average. At three it promises 0.0426 and the mean is 0.0454, but the worst function keeps 0.0796 — nearly double. At four functions the mean is 0.1713 against a promised 0.0791, more than twice it, and the worst function keeps 0.2772. At five and six the mean drifts back near the line, 0.1298 and 0.1289 against 0.1155 and 0.1496, while the worst function stays at 0.2381.

So at sixteen units the property the rule is chosen for — every function shrunk by the same factor — is a spread from mean to worst of up to two to one. That is a statement about small trials rather than about the rule; at a few hundred units the counting essay found the $\chi^2$ limit arriving for well-behaved functions. But a trial small enough to enumerate is exactly the trial in which rerandomisation constraints bind, and the theory is least reliable precisely where it is most needed.

Which to write down #

The two regions answer different questions, and the measurements here say which question each answers.

The box is the region for a basis whose functions are the point. If the experimenter has named the functions because each is a shape the outcome might have — which is how this field chose its bases — then the coordinates are not arbitrary, and a region that holds each coordinate is holding what was asked for. At the same count it does so twice as well in the worst function. Its cost is that it can be empty, and the counting essay showed where: the fourth function on sixteen units.

The ellipse is the region for a basis that is only a span. If the functions are a convenient parameterisation of a subspace — any polynomials up to degree three, say, which is a class with no worst case below its own dimension — then no coordinate deserves priority and the ellipse’s invariance is exactly right. It never runs out. Its cost is the one measured here: it gives the small, cancelling combinations of the basis the same weight as the large ones, which on correlated functions is a large share of its constraint.

The share a practitioner states is not the share they get. At sixteen units a 2.5% $\chi^2$ point admitted between 34 and 314 of a promised 322. The number to check is the admissible count itself, which on a trial this small can be computed exactly before the trial starts, and which fixes the resolution of the randomisation test afterwards.

On a trial this small none of it needs to be argued in the abstract. The whole space of equal splits can be walked in well under a second, so a protocol can state both regions, count both, and report for each the admissible count, the worst named function’s remaining imbalance and the finest p-value the resulting test can give. If the box is empty or nearly so, the choice is made; if both are comfortable, the box’s tighter worst case is usually worth its loss of invariance. What a protocol should not do is choose the ellipse for its theory and then never look at the count the theory was supposed to guarantee.

A hybrid follows directly from the direction figure: whiten only the directions with appreciable variance and leave the smallest ones out of the region, which is a Mahalanobis rule on a truncated span. It keeps the ellipse’s floor — never empty — and stops it spending on directions that move nothing. Whether it recovers the box’s tightness at a matched count is the open question below.

Still open: an ellipse on the directions that matter #

The natural repair drops the principal directions whose variance falls below some fraction of a single function’s, and draws the Mahalanobis region on the rest. At three functions on these units that would drop the direction carrying 0.219 of a function’s variance and keep the two carrying about one and two. The measurement is the same as here: the admissible count, the worst named function’s remaining imbalance at the box’s count, and how both move as the cutoff is varied from nothing to everything. It would say whether there is a region that never empties and still holds named functions as tightly as a box, or whether those two properties trade against each other at every cutoff.

What is claimed here, and what is not #

This essay takes the Mahalanobis acceptance rule against a box of per-function tolerances, on sixteen units, a basis of up to six functions from the field’s dictionary, and every one of the 12,870 equal splits. The claims are that at a stated share of 2.5% the Mahalanobis rule admits between 34 and 314 assignments at every basis size, never none, where the box admits none from four functions; that at the box’s own count it leaves about twice the box’s imbalance variance in the worst named function at three functions, and the same in the unnamed ones; that it does so by tightening a principal direction carrying a fifth of a function’s variance that the box leaves loose; and that the normal theory’s equal shrinkage holds at one and two functions and fails at four.

What stays out, named as a decision: larger trials, where the enumeration is unavailable and the counting essay’s estimates take over; other dictionaries, where the correlations between named functions and therefore the smallest direction would differ; and the analysis, which after either region is the randomisation test over the admissible set and is exact whichever shape the set has.

The boundary against the counting essay is the trial size and the question: it owns the share admitted at four hundred units and the span-invariance of the ellipse; this one owns what that invariance costs the named functions at a count small enough to enumerate.

The checks, and the refusal that makes them mean something #

Each figure checks its claim at every setting it can be dragged to: the Mahalanobis rule is never empty where the box is; at the box’s count the ellipse holds the worst named function less tightly; the box leaves the smallest principal direction looser and the largest tighter than the ellipse does; and the normal theory holds at one function and fails at four. The refusal is a single-function basis presented as if the two regions differed there — the box claimed to beat the ellipse at one function — which must fail, because at one function they are the same interval and admit the same assignments; a check that passed it would be measuring the ordering of ties rather than the shape of a region.