A share at a hundred units

Balancing a skewed covariate computed what a balancing rule removes of six outcome shapes when the covariates are not symmetric, and the table moved in two directions at once. The exact zeros a normal covariate gives filled in: a rule holding the means of two covariates removes nothing of their product on a normal covariate and 26.3% of it on one with a skewness of 2.26, and 52.0% of the square where on a normal covariate it removes nothing. And the guarantees a normal covariate gives weakened: the median splits’ hold on a linear shape fell from 65.7% to 48.9%.

Every one of those numbers is a projection — what a rule removes of a shape if it balances its dictionary exactly. The essay closed on the translation it did not make: a share is not an imbalance, and turning one into the other needs a design, a trial size and the covariate’s distribution together. This essay makes it, for the covariate-adaptive rule the rest of these essays use — the arm that leaves more information about the treatment effect under the dictionary’s model, taken with probability 0.8 — in trials of a hundred and four hundred units.

What a trial measures #

The outcome is not simulated, because it does not need to be. What a design does to an unadjusted estimate of the treatment effect is the imbalance it leaves between the arms in whatever function the outcome is built from, so the measurement is that imbalance: its variance under the rule, as a share of its variance under a coin, on the same units. A share of one is a coin. The closed form’s counterpart is one minus the share the projection removes, which is what an infinitely large trial holding the dictionary exactly would leave.

The covariates are two, correlated at a half through a Gaussian copula, with four marginals: normal; clearly skewed (skewness 2.26) and strongly skewed (4.75), both from Tukey’s exponential dial; and an exponential (2.00). The dictionaries are the means of the two covariates, and the means with the median splits. Each cell is four thousand trials.

On normal covariates the trial and the closed form agree about everything except the shape being balanced. The rule leaves the linear shape 4.3% of a coin’s imbalance at a hundred units and 1.1% at four hundred, where exact balance leaves nothing — the residual of a finite trial, falling like one over the trial size, as how many functions a trial can hold measured it. The median split keeps 39.5% at a hundred and 36.0% at four hundred against the closed form’s 36.3%; the mixed interaction keeps 77.9% and 76.8% against 75.0%. The square, the product and the product of splits stay at a coin’s, as parity says they must.

So on a normal covariate a share is an imbalance, give or take the few per cent the rule’s own residual costs the shapes it reaches. That is the baseline every other marginal is measured against.

The covariate the rule is holding #

The closed form says a rule holding the means leaves nothing of the means on every marginal, which is true by construction and is not the question. The trial says how close a hundred units get. On the normal covariate the rule leaves 4.3%; on the clearly skewed one 6.4%, on the exponential 5.9%, and on the strongly skewed one 11.1%. At four hundred units the four are 1.1%, 1.5%, 1.4% and 2.5%.

Both sizes say the same thing in the same proportion: multiply the residual by the trial size and it is about four on the normal covariate, about six on the clearly skewed and the exponential, and about ten on the strongly skewed. The rule balances a skewed mean at the same rate as a normal one, one over $n$, with a constant two and a half times as large.

The reason is in where a skewed covariate’s variance lives. A strongly skewed covariate’s mean is moved by its largest few values far more than by the rest, and a sequential rule can only respond to a large value when it arrives, by sending it to whichever arm it helps; it cannot undo the one before. So the residual is the residual on the few units the mean depends on, and a few units cannot be split as finely as many.

The product the closed form credits #

The most useful of the closed form’s skewed entries was the leak into the product. On a normal covariate a rule holding the means removes nothing of their interaction; on a skewed one the symmetry that made that an exact zero is gone and the closed form credits the same rule with between a quarter and three tenths of it.

The trial collects much less. On the clearly skewed covariates the closed form leaves 73.7% of a coin’s imbalance in the product — 26.3% removed — and a hundred-unit trial leaves 88.3%: 11.7% removed, under half the credit. At four hundred units it leaves 86.9%, so quadrupling the trial collects one and a half more points. On the strongly skewed covariates the closed form removes 29.7% and the trials 11.1% and 11.7%; on the exponential 28.5% against 16.2% and 16.6%.

The leak is real and it is slow. A shape a rule removes only by accident — because it is correlated with a function the rule holds, not because the rule holds it — is removed in a trial in proportion to how well the trial’s sample reproduces the correlation the closed form computed from the whole distribution. For a product of two skewed covariates that correlation is carried by the units in the joint upper tail, where both covariates are large together, and a trial of four hundred units at a correlation of a half has very few of those. The projection is a statement about the population; at a hundred units it is a statement about perhaps a dozen of its members.

The square shows the same thing, on a single covariate. On the clearly skewed marginal the closed form says the means remove 52.0% of the square; the trials remove 36.7% at a hundred units and 38.6% at four hundred. On the exponential, 80.1% against 68.7% and 74.4% — more of the credit collected, because an exponential’s tail is lighter than a lognormal-like one and fills faster.

Skewness is not the number that decides #

The exponential and the clearly skewed marginal have nearly the same skewness, 2.00 and 2.26, and the closed form treats them nearly alike: it credits the means with 28.5% and 26.3% of the product. The trials do not. At a hundred units the exponential collects 57% of its product credit and the clearly skewed covariate 44%; of the square’s credit, 86% against 71%.

What separates them is how far out the tail goes, not how lopsided the distribution is. Tukey’s dial at 0.6 is a transformed normal whose upper tail grows like an exponential of the latent variable, so its largest values in a sample of a hundred are much larger, in units of its own standard deviation, than an exponential’s. Skewness is a third moment and says how much asymmetry there is on average; the rate at which a trial collects a projection depends on how much of that asymmetry the trial’s largest handful of units carries, which is a fact about the tail beyond the third moment.

So the skewness a practitioner can read off their covariate is the right number for the question the earlier table answered — whether the zeros hold — and the wrong one for this question. Two covariates with the same skewness can have the same closed-form table and trials that collect it at very different rates. The one quantity in the table that is the same on every marginal is the median split’s residual, and it is the same precisely because a median split ignores the tail.

The median split, which the rule stops seeing #

The third shape is the one where the trial and the closed form part furthest, and they part in a direction that is easy to miss. A median split is correlated with its covariate on every marginal, so a rule holding the mean removes some of a median-split shape by accident, and the closed form says how much: 63.7% on a normal covariate, 47.3% on the clearly skewed one, 48.3% on the exponential, and 32.8% on the strongly skewed one.

On the first three the trials get close: they leave 60.5% removed on the normal covariate, then 40.5% and 42.0% at a hundred units, and much the same at four hundred. On the strongly skewed covariate a hundred-unit trial removes 19.9% of a median split, and a four-hundred-unit trial removes 1.8% — nothing, within the precision of four thousand trials. The rule’s hold on the bulk of the covariate gets worse as the trial gets larger.

That is the clearest statement of what balancing the mean of a strongly skewed covariate is. The larger the trial, the larger the largest values it contains, and the more of the mean’s variance those few values carry; the rule spends its choices sending them to alternate arms, and the units around the median — which decide a median split — are assigned for all practical purposes by the coin the rule takes with probability 0.2. A balance table would report the means beautifully balanced. The bulk of the covariate is balanced as a coin balances it.

And the rule that holds both #

A rule that holds the median splits as well as the means is the one the skewed covariate’s own table recommended, and the trials show why in a way the table could not.

On the strongly skewed covariates the median split is now held, and the trial leaves it 6.2% of a coin’s imbalance at a hundred units and 1.4% at four hundred — the same residual it has on every marginal, because a median split is a function of the sign of the latent normal and no transformation of the covariate moves it. The split is the one function in the dictionary whose balance a trial collects at the rate the normal theory promises whatever the marginal.

What it costs is the mean. Holding four functions with a hundred units, the rule leaves the linear shape 18.9% of a coin’s imbalance on the strongly skewed covariates, against 11.1% when it held the means alone and 6.4% on normal covariates holding the same four. At four hundred units the figure is 4.2%. The mean of a skewed covariate is the dear function to hold, the split the cheap one, and a dictionary holding both pays the mean’s price for both.

The product and the square fall short of the closed form in the same proportion as before: the closed form leaves 58.6% of the product, the trial 82.5% at a hundred units and 75.5% at four hundred; for the square, 46.8% against 73.4% and 61.2%. With the splits in the dictionary the trial collects more of each as it grows, and at four hundred units it has collected three fifths of the product’s credit and three quarters of the square’s.

What a trial on a skewed covariate can rely on #

Three readings come out of the table, and they rank the closed form’s entries by how much of them a trial actually gets.

What the rule holds, it gets — at a price that grows with the skew. The functions in the dictionary are balanced at the usual rate of one over the trial size, and the constant is set by how much of each function’s variance a few units carry: about four for a mean on a normal covariate, ten on a strongly skewed one, and the same small constant for a median split on every marginal.

What the rule removes by symmetry, it gets exactly. The zeros of the earlier essays and their absence are about parity, and parity is a property of the latent normal that a trial reproduces unit by unit. The product of the median splits, which no dictionary here reaches, keeps a coin’s imbalance on every marginal and at both sizes, as the closed form says it must.

What the rule removes by correlation, it gets slowly and partly. Every share that a skewed covariate earns because its symmetry has gone — the leak into the product and the square, the correlation of a mean with its own split — rests on the tail, and a trial collects it only as fast as its tail fills in. At four hundred units that is about half of the credit for the product and three quarters for the square on the clearly skewed covariate; on the strongly skewed covariate’s median split it is none of it.

The third is the one that changes what the earlier table is worth to a designer. It found the advice to hold an even function worth between 1.32% and 4.60% on skewed covariates, and those numbers are projections of exactly this third kind. On the evidence here a trial of a hundred units would collect perhaps half of each.

Thirty-two cells of four thousand trials #

Four thousand trials in each of thirty-two cells: four marginals, two dictionaries, two trial sizes. On every trial the same units are assigned by a coin and by the rule, and the imbalance in each shape is the difference between the arms’ means of that shape, so each share is a ratio of two variances over the same units. The interaction shapes are products of centred factors, centred at the factors’ exact means from quadrature, which is the definition the closed form uses. A share near one has a sampling error of about three points, which is why the product of the median splits wanders between 91% and 101% around a closed form of 98% to 100%; every claim above rests on a difference several times that.

The closed-form column is the field’s own nested quadrature, unchanged. The rule is the one how many functions a trial can hold measured with one covariate, here holding two or four columns, and its criterion is unchanged by rescaling a column, so no function needed standardising.

A skewed covariate’s share is earned slowly, and only partly #

The claims are that on skewed covariates a covariate-adaptive rule holding the means balances them to a residual two and a half times a normal covariate’s at the same trial size; that it removes about half of the product’s share the closed form credits at a hundred units and barely more at four hundred; that it removes about seven tenths of the square’s on the clearly skewed covariate; that on the strongly skewed covariate it removes a fifth of a median split at a hundred units and nothing of it at four hundred; and that adding the median splits restores the split at the normal rate on every marginal and makes the mean dearer.

Not claimed: anything about thresholds away from the median, which are neither odd nor even and would sit between the split and the mean; anything about rules that read a transformed covariate — a log, a rank — which would turn a skewed covariate back into a symmetric one and is the obvious repair; anything about correlations other than a half; and anything about the analysis, which can adjust for a shape the design left unbalanced and is a separate lever.

Still open: balancing the covariate on its own ranks #

A rule that balances the ranks of a covariate rather than its values balances a function whose distribution is uniform whatever the marginal, so every symmetry argument holds for it exactly and its residual constant is a normal covariate’s. It gives up the tail: a rank treats the largest value as one more unit. Whether that trade wins — whether a rule holding the ranks and the median splits leaves the linear outcome, the product and the square closer to the closed form on a strongly skewed covariate than a rule holding the values, at a hundred units and at four hundred — is the measurement that would say whether the skew is a property of the covariate or of the scale the rule was told to read it on.