The fourth moment that was missing

A balancing rule is handed a list of functions of the covariates and refuses any assignment whose imbalance in them is too large. What that removes from an outcome of a given shape is a projection, so the whole subject is geometry in the space of square-integrable functions — and the geometry is exact whenever the inner products are.

For one normal covariate they are: the orthonormal Hermite functions diagonalise everything, and a threshold’s inner products with them have closed forms. For two independent normal covariates they still are, because a product term’s inner product factorises into two one-dimensional ones and every factor is already known.

For two correlated covariates the factorisation is false, and the essay that built the product dictionary says exactly where it breaks. Mehler’s formula gives

$$\operatorname{E}[h_i(X)\,h_j(Y)] = \begin{cases} \rho^{\,i} & i = j \\ 0 & i \neq j \end{cases}$$

for a standard bivariate normal pair, which makes the main-effect Gram matrix exact at any correlation. The interactions need E[h_i(X)h_k(X) · h_j(Y)h_l(Y)], which is an expectation of four Hermite functions and is not that formula. That expectation is what was missing.

Turning four into two #

The missing piece is not an integral. It is an algebraic identity about polynomials, and it has been available since the nineteenth century.

A product of two Hermite polynomials of the same variable is again a combination of Hermite polynomials of that variable:

$$He_i\,He_k = \sum_r \binom{i}{r}\binom{k}{r}\, r!\; He_{i+k-2r},$$

In the orthonormal scaling h_j = He_j/√(j!) that becomes h_i·h_k = Σ_m L(i,k,m) h_m with

$$L(i,k,m) = \binom{i}{r}\binom{k}{r}\, r!\, \frac{\sqrt{m!}}{\sqrt{i!\,k!}}, \qquad r = \frac{i + k - m}{2},$$

and L is zero unless m has the same parity as i + k and lies between |i − k| and i + k.

With that, the fourth-order expectation is a second-order one. Write F = f·g and G = u·v as Hermite series in their own variable — which the linearisation says is always possible — and apply Mehler to F and G:

$$\langle f(X)u(Y),\; g(X)v(Y)\rangle = \sum_m (fg)_m\,(uv)_m\,\rho^{\,m}.$$

That is the whole thing. Every inner product in a product dictionary at any correlation is a finite sum of products of two Hermite coefficient sequences, weighted by powers of ρ.

It is worth pausing on how little that expression assumes. There is no truncation in it for polynomial factors, no numerical integration, no requirement that the two functions be of the same kind. A threshold multiplied by a cubic, against a quadratic multiplied by a threshold, at a correlation of 0.63, is a finite sum of products of coefficients — and the only thing standing between it and being exact is whether the threshold’s own coefficient sequence terminates.

And the sum is dominated by its low-order terms whenever ρ is not close to one, because the weight is ρ^m. At ρ = 0.5 the tenth-order term carries a factor of one in a thousand, so an expression that is in principle infinite is in practice a handful of terms. That is a property of Mehler’s weighting rather than of any approximation made here, and it is why the whole construction is cheap enough to sit inside a search over subsets.

Two checks it has to pass #

The first is a special case. At ρ = 0 only the m = 0 term survives, and the zeroth coefficient of a product fg is Σ_i f_i g_i, which is ⟨f, g⟩. So the expression collapses to ⟨f,g⟩⟨u,v⟩ — the factorisation the independent case rests on. The general form contains the one it generalises, and that is checked across every pair in the twenty-four-term dictionary rather than argued: the worst departure is four parts in ten thousand million million.

The second is against draws.

Two hundred thousand draws of a bivariate normal pair at ρ = 0.5, every pair of dictionary entries evaluated on each draw and averaged. The worst departure over thirty-six pairs is 1.99 standard errors — measured in each pair’s own error, because the entries differ in size by two orders of magnitude and one tolerance would be a false alarm on the small ones and a rubber stamp on the large.

Neither check can confirm itself. The closed form never touches a random number and the count never touches a factorial.

There is a third check available and it is the cheapest of the three: norms. Every dictionary entry is scaled to unit variance at ρ = 0, and the closed form has to reproduce that — and, more usefully, has to say correctly how the norm changes as the covariates move together. A pure interaction h₁(x)h₁(y) has squared norm 1 + 2ρ², so it stops being a unit vector at any correlation, and a routine that quietly assumed unit norms would be wrong by half at ρ = 0.5 without producing anything that looked like an error. The projections below divide by the norm the closed form gives rather than by one, and that division is the difference between a share of a shape and a number with no interpretation.

Where it stops #

The identity is exact for polynomials and it is not exact for everything, and the boundary is sharp enough to be worth drawing.

A polynomial of degree d has finitely many Hermite coefficients, so a truncation past d is not an approximation at all, and a product of two of them is exact past 2d. Sixteen orders is therefore exact for a dictionary of powers up to the fourth, rather than adequate for it.

A threshold is a different object. The indicator 1{x > c} has an infinite Hermite expansion whose coefficients decay like $j^{-3/4}$, so the tail of the squared coefficients falls like 1/√J — and a tail that falls like the square root of the truncation is a tail that does not go away.

At ten orders a median split has 16.18% of its variance outside; at sixty, 6.57%; at a hundred and sixty, 4.02%. A cut at two is worse at every truncation.

So the dictionary here carries a fourth power where the one-covariate work carried a median split. That is a substitution, and it is not a cosmetic one: it is the difference between geometry that is closed and geometry that is asymptotic, and stating it is the only way the numbers below mean what they say.

Where the closed form exists, it exists exactly. Where it does not, it does not exist a little less than exactly. A threshold’s geometry at a correlation is available to whatever precision somebody is willing to pay for in orders, and it is a different kind of claim.

The one thing the substitution changes #

Replacing a median split with a fourth power in the dictionary is not free, and it is worth being explicit about what it costs rather than presenting it as a technicality.

A median split is a realistic thing to hand a balancing rule. Trials stratify on binary and categorical covariates constantly, and a cut of a continuous one is the commonest way a continuous covariate enters a randomisation scheme at all. A fourth power is not something anybody hands a rule on purpose; it is in the dictionary because a dictionary of powers is what an orthogonal polynomial expansion is.

So the dictionary here is a model of a real dictionary rather than a real one, and the results below are statements about the geometry rather than recommendations about what to balance on. What transfers is the mechanism — a correlation between covariates makes an interaction acquire components along both covariates’ higher powers, and the size of the component is a closed function of ρ — and what does not transfer is any particular number attached to a particular cut point.

The essay on cutting a continuous covariate measures what a cut costs in the one-covariate world, and the same tail that makes the geometry asymptotic here is what makes that cost large there. The two are the same fact about thresholds seen from opposite directions: a cut throws away a great deal of a covariate, and its Hermite series converges slowly, because both are statements about how badly a step function is approximated by smooth ones.

What it buys #

The point of a closed Gram matrix is not the Gram matrix. It is that everything built on it becomes arithmetic.

What a rule removes of a shape is that shape’s squared multiple correlation on the span of what the rule reads, which is a′G⁻¹a with G the Gram matrix — so it is now available at any correlation without simulation.

The maximin basis over a list of shapes is a finite search over subsets, scoring each by its worst guarantee, and every score is one of those projections. It moves as ρ moves, because its inputs do.

What a rule gets for nothing on the second covariate is Mehler’s ρ^j pairing squared, which is ρ²ʲ — at a correlation of a half, 25.000% of the linear term, 6.250% of the quadratic, 1.563% of the cubic and 0.391% of the fourth. Geometric in the order, so real and quickly exhausted.

And the thing the whole construction was built to test is the next essay: what a rule holding every main effect of both covariates removes of a pure interaction. It is exactly nothing when the covariates are independent, which is a result that has been quoted as though it were about balancing rules. It is about independence.

What moves and what does not #

Two things about the geometry are worth separating before any of it is used, because one of them survives the correlation intact and the other does not.

What survives: a balancing rule still cannot tell one basis from another with the same span. What a rule removes of a shape is the shape’s squared multiple correlation on the span of what the rule reads, and a squared multiple correlation is a property of the subspace. Rotating within the span changes the functions and changes nothing about what the rule removes. That was established for one covariate and it is a statement about projections rather than about covariates, so the correlation does not touch it.

What does not survive: every number the projection produces. The Gram matrix’s entries all move, the projections built from them all move, and the ordering of which basis is best over a list of shapes moves with them. So the theorem is stable and its outputs are not, which is the most dangerous combination available — a result that keeps being true while every number it produced goes stale.

The practical form of that is short. A basis chosen by scoring subsets at ρ = 0 is not the basis that scores best at ρ = 0.5, and the covariates in a real trial are correlated. Choosing one at the wrong correlation is not a violation of any theorem; it is arithmetic done on the wrong inputs.

What a worst-of-thirty-six is allowed to be #

“Within two standard errors over thirty-six pairs” is quoted as a pass, and it is worth asking what a correct implementation would have produced, because the answer is not two.

The maximum of thirty-six independent standardised departures sits at about 2.34 standard errors half the time, and exceeds 2.0 about four times in five. So a gate written as the worst of thirty-six entries is within two standard errors is a gate a correct implementation would fail more often than it passed, if the thirty-six were independent.

They are not, and the counted worst of 1.99 says by how much. For a maximum to sit at 2.0 typically, the entries have to behave like about fifteen independent ones rather than thirty-six — which is what sharing two hundred thousand draws between every pair of a symmetric eight-by-eight Gram matrix does to them.

That makes the check stronger than its wording, not weaker. The observed worst is below the median of what fifteen effective entries would give, so the agreement is comfortable rather than marginal — and the reason to say so is that the same tolerance applied to a larger dictionary, or to entries computed on independent draws, would start rejecting correct code. A threshold in standard errors is a threshold on a maximum, and a maximum’s distribution depends on how many things it is a maximum of and how alike they are — which is the same arithmetic this site’s set-comparison fields spend essays on, arriving inside one of its own gates.

How deep the correlation reaches depends on the correlation #

The free shares — ρ²ʲ at 25.0%, 6.25%, 1.56% and 0.39% — are read as geometric and quickly exhausted, and both halves of that are true at ρ = 0.5 and only there.

At a correlation of a half the fourth order is worth 0.4% and the sixth would be 0.02%, so a dictionary deeper than about four orders gains nothing from the correlation at all. At a correlation of 0.8 the same sequence is 64%, 41%, 26%, 17% — and the sixth order is still 6.9%, which is more than the first order is worth at ρ = 0.5.

So “geometric and quickly exhausted” is a statement about the base of the geometric rather than about the shape. The number of orders the correlation reaches into goes as 1/log(1/ρ²), which is 1.4 orders per decade of decay at ρ = 0.5 and 5.2 at ρ = 0.8 — nearly four times as deep for a correlation less than twice as large.

The practical reading is about where the closed form earns its keep. At weak correlations a dictionary could be scored at ρ = 0 and corrected by hand for the first order or two. At strong ones the correction reaches further than any dictionary anybody writes down, and every entry of the Gram matrix is doing work — which is precisely the regime a real trial’s covariates sit in, and the regime the linearisation was built for.

Why an expectation and not a simulation #

There is an obvious objection to all of this, which is that a Gram matrix can be estimated from draws in a line of code and nobody needs a linearisation.

Three answers, and the third is the one that matters.

An estimated Gram matrix is inverted, and inverting an estimate is where small errors become large ones. The entries here differ in size by two orders of magnitude and several are near-cancellations; a two-standard-error wobble in an entry that is nearly zero is a much larger wobble in the projection it feeds.

A maximin basis is a minimum over shapes of a quantity that is itself a minimum, and minima of noisy quantities are biased. A search over seven hundred and thirty-five thousand subsets scored by estimates will find the subset whose estimate happened to be lucky.

And a closed form can be checked. An estimated Gram matrix agrees with itself; a closed one and a counted one are two routes that share no arithmetic, and the only reason to believe either is that they agree when neither could have been made to.

What is claimed here, and what is not #

This essay takes the expectation that made a product dictionary’s geometry unavailable at a correlation, and the claims are that a product of two Hermite functions is a finite combination of Hermite functions with closed coefficients, that the fourth-order expectation is therefore Mehler applied to the two products, that the general form reduces to the independent-covariate factorisation at ρ = 0 to machine precision, and that it agrees with two hundred thousand draws to within two standard errors over thirty-six pairs.

What stays out and is named as a decision: thresholds, whose Hermite expansion is infinite and whose truncation tail falls like 1/√J, so every claim about them here is asymptotic and is drawn as such; more than two covariates, where the same linearisation applies and the dictionary grows as a power; and any non-normal marginal, since Mehler’s formula is a statement about the bivariate normal and there is no route to it otherwise.

The boundary against the dictionary field is that it builds the dictionary and this one supplies the inner product the dictionary needs once the covariates move together.

The checks, and the refusals that make them mean something #

Two claims are gated in this field’s library. The general inner product is required to equal the independent-case factorisation at ρ = 0 over every pair in the dictionary, which is an identity and would break under any error in the linearisation coefficients. And every entry is required to agree with two hundred thousand draws in its own standard error, interaction-by-interaction entries included — those being precisely the ones the closed form was unavailable for.

The refusal in this field is the interaction result quoted for correlated covariates. It is correct and it is stated for independence, and carried across it says a rule protects nothing of an interaction where in fact it removes a third of one — an error in the direction that flatters the rule.