A basis drawn rather than chosen

A balancing rule on two covariates has twenty-four functions it could hold — four of each covariate alone and sixteen products — and seven shapes the outcome might take: linear, quadratic or a median split in the first covariate; linear in both, added; their product; the first covariate above the second’s median; and both above their medians. Where the enumeration stops found the best fixed basis of each size by walking every subset. Its first finding is the one this essay starts from: with one function or two, the best basis guarantees exactly zero. Two of the seven shapes are pure interactions, two more are main effects of different covariates, and no one or two functions touch all of them. The smallest basis that protects every shape at all has three functions and removes 42.44% of the worst.

That essay named the alternative and set it aside — randomising the basis, which the one-covariate field measures and which would need a distribution over twenty-four terms rather than eight. The one-covariate field found that drawing the basis at random from a least favourable mixture equalises the shapes and more than doubles the guarantee. Twenty-four terms turn out to be no obstacle: the subsets number 24, 276, 2,024 and 10,626 at one to four functions, the payoff for each against each shape is an exact inner product, and the game can be solved in seconds.

The game, and its two answers #

The experimenter chooses a basis of k functions; nature chooses a shape; the payoff is the share of the shape the basis removes, exactly, from the product inner product the field built. A fixed basis is one row of that table. A drawn basis is a probability distribution over rows, sampled once before the trial and then held. Fictitious play — each side repeatedly best-responding to the other’s history — converges to both players’ optimal mixtures, and the gap between what the experimenter’s mixture guarantees and nature’s best reply bounds how far the answer is from the game’s value. After twenty thousand rounds that gap is 0.0009, 0.0014, 0.0009 and 0.0011 at one to four functions.

The drawn basis guarantees 0.2042 of the worst shape at one function, 0.4084 at two, 0.6133 at three and 0.7920 at four. The fixed basis guarantees 0, 0, 0.4244 and 0.6366. Where the fixed basis has nothing to offer, the drawn one offers a fifth of a shape per function — the first three steps are 0.2042, 0.2042 and 0.2049 — and at three and four functions it is 1.45 and 1.24 times the fixed basis’s guarantee.

Why the answer can be trusted #

Fictitious play is a slow algorithm with one great virtue: it certifies itself. At every round the experimenter’s accumulated mixture has a worst case over the shapes, which is a guarantee it actually achieves; and nature’s accumulated mixture has a best reply over the bases, which no basis can beat against it. The game’s value lies between the two numbers, whatever the algorithm has or has not converged to. After twenty thousand rounds they are 0.2042 and 0.2051 at one function, 0.4084 and 0.4098 at two, 0.6133 and 0.6142 at three and 0.7920 and 0.7931 at four. Every guarantee quoted here is the lower of each pair, so each is achieved by a mixture that exists, and none is more than a seventh of a point below the best possible.

The payoffs themselves are not estimated either. Each is the squared multiple correlation of a shape on the span of a basis, computed from the product inner product of normalised Hermite functions and indicators, which the dictionary essay checked against three hundred thousand draws. The only approximation anywhere in the game is the number of rounds, and the bracket measures it.

Two shapes nature never plays #

Why a fifth of a shape per function #

The linear rate has a short explanation, read off nature’s side of the game.

At one, two and three functions nature puts the same weights on the shapes, to within 0.002: 20.5% on the quadratic shape, 25.0% on the additive one, 12.6% on the product, 4.4% on the moderated shape and 37.6% on the corner, and nothing on the linear shape or the median split. Against that mixture, the most any single function can remove on average is about a fifth — the quadratic term removes all of the quadratic shape and nothing else, worth exactly its 20.5% weight, and every other function’s best is no better. A basis of k functions whose members do not overlap in what they remove gets k fifths, and the drawn mixture is built from exactly such bases, so the guarantee grows by a fifth a function until there is no longer room to avoid overlap.

At four functions that room runs out and nature shifts. The quadratic shape falls to 9.2% and the corner rises to 42.8%: with four functions a basis can afford a squared term without giving anything else up, and the quadratic shape stops being where the guarantee is lost. The fourth step is worth 0.1787 rather than a fifth.

Nature never plays the linear shape or the median split of the first covariate, at any size, and the reason is dominance. Any basis that removes much of the additive shape — linear in both covariates — removes the linear part of the first covariate with it, and any basis that holds the median split of the first covariate holds part of the corner shape’s main effect too. A shape that every good basis protects as a side effect is not a shape nature can win on, so it carries no weight, and the guarantee is set entirely by the five shapes that can be missed.

The corner shape — both covariates above their medians — is nature’s heaviest weight throughout, and the reason is in the dictionary essay: it is an interaction with two main effects inside it, so no single function holds more than a third of it and a basis has to spend on all three parts to protect it fully.

What the mixture draws #

The experimenter’s side of the game is a distribution over bases, and its shape says what kind of protection is on offer.

At two functions the mixture draws twelve bases with weight above half a per cent, and none of them more than 13.6% of the time: the median split of both covariates, at 13.6%; the linear term of both, at 12.1%; the square and the median split of the first, at 9.7%; the square of the first with the product of the two, at 9.5%; and so on down. Between 27% and 30% of the functions drawn are interactions, at every size from one to four. The fixed maximin basis at three functions — the linear and squared terms of the first covariate and the first above the second’s median — is one of the thirteen bases the mixture draws at that size, at 8.0%.

That spread is the mixture’s whole mechanism. Nature cannot pick a shape the trial will miss, because the trial has not yet decided which shapes it will hold. It is also the mixture’s whole cost.

The guarantee is an average over draws #

A trial does not balance a mixture. It draws one basis before the first unit arrives and balances that basis for the rest of the trial. The guarantee of 0.6133 at three functions is an average over the draw: what a trial can expect before it knows which basis it got. Once it knows, it has one row of the table.

At three functions the drawn basis removes nothing of the quadratic shape in 38.7% of draws, nothing of the product in 18.8% and nothing of the moderated shape in 18.8%. The fixed basis of three functions removes at least 42.44% of every shape in every trial. So the comparison the two guarantees invite — 0.6133 against 0.4244 — is a comparison between an expectation and a floor, and which one a trial should want depends on what the trial is for.

The exposure falls with the size of the basis but is never small. At one function the worst shape goes wholly unprotected in 79.5% of draws; at two in 59.1%; at three in 38.7%; at four in 20.4%, where the fixed basis’s floor has risen to 63.66%.

What a drawn basis asks of the analysis #

A basis drawn at random is part of the randomisation, and the analysis has to know it. A randomisation test compares the observed statistic with its value under the assignments the design could have produced, and a design that draws a basis and then balances it could have produced the assignments of every basis it might have drawn. Two reference sets are defensible: the assignments admissible under the basis actually drawn, which conditions on the draw and is exact for that reason; or the assignments admissible under any basis, weighted by the mixture, which is the design’s full reference distribution. What is not defensible is a reference set built from complete randomisation, which describes a trial that never ran — the error the essay on randomisation and balance traces in its simplest form, and which the experiments that could have happened repairs by construction.

The conditional choice is the practical one, and it has a pleasant property here: conditional on the draw, the trial is an ordinary balanced trial on a fixed basis, and everything this field has measured about fixed bases applies to it unchanged. The draw changes what the experimenter can promise before the trial. It does not change how the trial is analysed afterwards, as long as the draw is recorded.

Which a trial should want #

A single trial should usually want the floor. A trial is run once. If the outcome turns out to be quadratic in the first covariate, a fixed three-function basis has removed all of that shape’s imbalance and a drawn one has removed all of it in three trials of five and none of it in two. The expected protection is higher for the drawn basis and the protection this trial actually gets is a coin toss the experimenter cannot see until the outcome arrives.

The comparison is not the same for every shape, and that is the part a protocol can use. The fixed three-function basis removes all of the quadratic shape and all of the moderated one; the drawn basis removes 0.613 and 0.617 of them on average, all or nothing draw by draw. For the corner shape the order reverses: the fixed basis removes 0.424 of it in every trial and the drawn basis 0.613 on average, and no draw at that size leaves the corner wholly unprotected. An experimenter who fears the corner shape most — two thresholds acting together, which is what a clinical eligibility rule looks like — does worse by drawing in only 6.4% of draws, and then by 0.077 of the shape, and does half again better on average.

A programme of trials can want the expectation. Across many trials balancing the same covariates — a platform trial, a network running the same protocol at many sites — the drawn basis’s average is the average a reviewer pooling them would see, and it is half again the fixed basis’s floor. That is the setting in which the design field’s maximin recommends randomisation, and the arithmetic here agrees.

And below three functions there is no choice to make. The fixed basis guarantees nothing at one or two functions, so the only guarantee available at all is the drawn one’s. A trial that can afford only two constraints — a small trial, where the constraints run out quickly — either draws its basis or protects nothing in the worst case.

The guarantee in expectation is not a property of the trial’s balance but of the experimenter’s ignorance about which shape will matter, and it is honest only if the basis really is drawn — by a mechanism fixed before the covariates are seen, recorded, and not re-drawn when the first draw looks inconvenient. A basis “chosen at random” by someone who could see the shapes is a fixed basis chosen badly.

How this differs from the one-covariate case #

The one-covariate field found a drawn basis worth more than twice the best fixed one at two functions, and an equalised payoff across the shapes the mixture plays. Two covariates change both. The equalisation holds — at three functions the drawn basis removes 0.613 to 0.617 of every shape nature puts weight on — but the ratio to the fixed basis is undefined at one and two functions, because the fixed basis is at zero, and falls to 1.45 and 1.24 at three and four. With one covariate, three functions of one number could protect every shape that field listed because every shape was a function of the same variable and the dictionary’s functions overlapped heavily in what they removed. Two covariates break the overlap: a function of the first covariate removes nothing of a function of the second, and an interaction removes nothing of either main effect. The interactions are what make the fixed basis worthless at small sizes, and they are also what make the drawn basis’s per-trial exposure large: a basis without an interaction misses the product and moderated shapes entirely, and most draws at small sizes have none.

Still open: a draw constrained to keep a floor #

The two answers here are the ends of a range. The fixed basis maximises the floor; the drawn one maximises the expectation with no floor at all. In between are mixtures restricted to bases that each remove at least some stated share of every shape — at three functions, every basis that removes at least a quarter of each shape, say, which excludes all the bases that leave the quadratic shape untouched. Solving the game over that restricted set gives the expectation a trial can have while keeping a floor of its choosing, and tracing it as the floor rises from zero to the fixed basis’s 42.44% gives the whole trade between the two in one curve. The same game, solved over the restricted set, computes every point on it; what it needs is somebody to say where on that curve a single trial should sit.

What is claimed here, and what is not #

This essay takes the randomised maximin basis over the two-covariate product dictionary. The claims are that against seven shapes the best fixed basis guarantees 0, 0, 0.4244 and 0.6366 of the worst shape at one to four functions; that a basis drawn from the least favourable mixture guarantees 0.2042, 0.4084, 0.6133 and 0.7920 in expectation, each within 0.0015 of the game’s value; that nature’s mixture is the same to within 0.002 from one function to three and shifts at four; and that at three functions the drawn basis leaves the quadratic shape wholly unprotected in 38.7% of draws, where the fixed basis protects every shape by at least 42.44% in every trial.

What stays out, named as a decision: the constrained draw, deferred above; the randomisation cost of either basis — how many admissible assignments it leaves, which the counting essay measures for fixed bases and which a drawn basis inherits draw by draw; and other shape lists, since every number here is conditional on the seven, which is the judgement this field has always named rather than made.

The checks, and the refusal that makes them mean something #

The headline figure checks that the drawn basis beats the fixed one at every size and that the game is solved to within half a point; the risk figure that the drawn basis leaves some shape wholly unprotected in more than a tenth of draws; the support figure that the mixture draws bases with and without interactions; the nature figure that its mixture is stable from one function to three. The refusal is a fixed basis presented as a drawn one — the pure maximin basis at three functions credited with the drawn basis’s guarantee of 0.6133 — which must fail, because a fixed basis’s worst case is a floor and cannot exceed what the best fixed basis achieves.