The family behind the letters

The optimality field measured four criteria and ended by naming the ones it had not: E-optimality, the Φₚ family it belongs to, and Ds-optimality. This essay takes the first two, and the reason to take them together is that they are the same thing — E is not a fifth letter to add to a list, it is the end of a continuum whose other named points are already on the list.

The continuum has one dial:

Φₚ(M) = [ (1/p′)·tr(M⁻ᵖ) ]^(1/p), minimised

where M is the information matrix and p′ its size. Every member is a power mean of the eigenvalues of M⁻¹ — the variances of the estimated parameters, in the directions the design happens to make independent — and p decides how the mean is taken. At p = 1 it is the arithmetic mean, which is A-optimality. As p → 0 it becomes the geometric mean, which is D-optimality. As p → ∞ it becomes the maximum, which is E-optimality, because the largest eigenvalue of M⁻¹ is the reciprocal of the smallest of M.

So the choice between the letters is a choice about how much weight to put on the worst-determined direction, and nothing else at all.

One algorithm for the whole family #

The optimality field built its search from the general equivalence theorem: put weight where the criterion’s directional derivative is largest, renormalise, repeat, and stop when the largest derivative reaches its own average. Three criteria were implemented that way, each with its own derivative worked out separately.

The family collapses those three into one:

δᵢ = fᵢ′M⁻⁽ᵖ⁺¹⁾fᵢ, against an average of tr(M⁻ᵖ)

and setting p to 0, 1 or ∞ reproduces the three derivatives the earlier field derived one at a time. That is not a tidying-up. It is a claim, and it is checkable: this file’s Φ₁ answer and the optimality field’s A-optimal measure are computed by two implementations that share no code below the information matrix itself, and they are required to agree to five decimal places. So are the p → 0 answer and the D-optimal measure.

The check is there because the fleet has a standing warning about exactly this situation. A shared name is not a shared function — the consolidation notes record three implementations of gauss that turned out to be three different functions sharing a word — and “Φ₁ is A-optimality” is a sentence of that kind until somebody makes the two run against each other.

What the dial does #

Run the dial and two things happen at once, in opposite directions.

The smallest eigenvalue rises. At the D end the optimal design’s information matrix has a smallest eigenvalue of 0.09927. At p = 1 it is 0.16516. At p = 64, effectively E, it is 0.19943 — twice what the D-optimal design achieves. That is the family doing what it was asked to: the further along the dial, the more the criterion cares about the direction the design knows least about, and the more of the experiment gets spent shoring it up.

The gap above it closes. The distance from the smallest eigenvalue to the next one runs 0.06105 at the D end, 0.03035 at A, and 0.00115 at p = 64 — a fifty-three-fold collapse.

The second is the interesting one, and it is not incidental. A smallest eigenvalue is not a differentiable function of a matrix where it is repeated: two eigenvalues crossing produce a corner, not a slope, and the maximum of a minimum generally sits exactly at such a corner. The family is driving the gap to zero because that is where its own optimum is.

So the one criterion in the family whose meaning fits in a sentence — make the worst-determined direction as well determined as possible — is the one whose optimum sits on a corner of its own surface. Every other member is differentiable, has a directional derivative, and yields to the algorithm in a few hundred iterations. E is reached as a limit, at p = 64, with an equivalence gap of a thousandth rather than the 10⁻¹¹ the differentiable members reach.

This is the mirror image of the optimality field’s own finding about D. There, the theorem was an equality and was gated at machine precision because a loose tolerance would have accepted an answer ten orders of magnitude worse. Here the theorem is not available at all at the endpoint, and saying that plainly is better than quoting a number that looks like the others.

Is that the family, or is it the square #

There is a reason to be suspicious of both movements, and the optimality field supplied it. Its last essay found that the region reverses the ranking of two catalogue designs on all four criteria at once — a disagreement larger than any disagreement between the criteria themselves — and the region never appears in a design’s name. A property of the Φₚ family established on a square and nowhere else would be a much smaller claim than the one made above.

So the same computation runs on a circular region, where the corners that carry most of the D-optimal design’s weight are simply not available. The smallest eigenvalue rises from 0.0508 to 0.0994 — again a doubling — and the gap falls from 0.0533 to 0.0009, again by a factor in the tens. The absolute values move, because a disc is a smaller region and carries less information per run. The two movements do not.

That is the difference between “on this region the family behaves like this” and “the family behaves like this”. Neither region is the general case and two is not a proof, but a claim that survives the one change known to overturn conclusions in this field has earned more than a claim that has only been made once.

The letters are not interchangeable, and the size of that is the point #

If the family’s members all returned nearly the same design, the whole thing would be a curiosity. They do not.

The D-optimal design is 50.2% efficient on E. Half the information in the worst-determined direction, given away by a design that is called optimal and is optimal for a different question.

The reverse is nothing like as bad. The E-optimal design is 95.3% efficient on D — it gives up under five per cent of the determinant. And A, which sits between them, is 98.67% on D and 83.53% on E.

That asymmetry is worth stating carefully because it is the only piece of advice this essay contains. A design chosen for the volume of its confidence region can be catastrophic in one direction and nothing in the volume reports it, because a volume is a product and a product is insensitive to one small factor being made up for by others. A design chosen for the worst direction is nearly optimal on volume as well, because fixing the worst direction costs little where the others were already fine. The letters are not symmetric, and if one of them had to be chosen blind, it would not be the one that is chosen by default.

Where the runs go #

The three designs are supported on the same nine settings — the 3² factorial, which is what the optimality field found for D and what the search keeps returning whatever the criterion — and they differ entirely in the weights.

The direction of the deformation says what the worst-determined direction is. As p rises, weight moves from the corners to the centre — so the direction the D-optimal design knows least about is one the centre informs, which for a quadratic model is the intercept-and-curvature combination that only a centre run can separate. E is buying that separation, and paying for it in the interaction.

That should sound familiar, because it is the same separation the response-surface field’s first essay is about. A two-level factorial cannot see a curve at all: at every corner both squared terms equal one, the column that would estimate curvature is a copy of the intercept, and the information matrix is singular. Centre runs buy exactly one number back. What the family shows is that the same combination remains the weakest direction even in a design that can estimate it — adding centre runs makes it estimable and does not make it well estimated, and E is the criterion that notices.

There is a second thing worth reading off these three pictures, and it is a check rather than a finding. All three measures are supported on the same nine settings, so nothing about the disagreement between the criteria is about which settings to visit. The optimality field established that the search returns the 3² factorial for D over 121 candidates; it returns the same nine for A and for E. The letter decides the weights and not the design, on this model and this region, and that is a narrower and more useful statement than “different criteria give different designs”.

On the region, and on the designs a catalogue offers #

The optimality field’s last essay showed that the choice of region reverses the ranking of two catalogue designs on all four criteria at once. Adding E and Ds to that table does not change the conclusion and does sharpen one row.

The design that wins the first column being nearly worst in the fifth is the sharpest form of this field’s argument. Both columns are efficiencies against their own optima, both are computed from the same information matrix, and the letter that decides which is reported is a choice made — almost always — by whichever letter comes first alphabetically.

One row of that table needs its own sentence, because it is the reason the table has an E column at all. The thirteen-run exchange design is the one the optimality field found by search, and by its own criterion it is excellent: 0.998 on D, which is as close to the theoretical optimum as an integer number of runs can get — better on E than only the rotatable composite’s 0.386, and worse than the 3² factorial’s 0.557. The same field showed that thirteen runs divide into the weights better than any other size in the range. That design is 0.439 on E. Searching hard for the best design under one criterion produced the worst row under another, and the search cannot be blamed — it did exactly what it was told, over 121 candidates, and reported its answer with an efficiency attached.

The asymmetry, as a maximin choice #

The essay’s one piece of advice is that the letters are not symmetric, and the cross-efficiencies price it exactly.

A design chosen for D and scored on E gives up 49.8 points. A design chosen for E and scored on D gives up 4.7. That is a ratio of 10.6, and it means the two choices carry very different exposures if the question turns out to be the other one.

Read as a maximin, the three named designs have worst efficiencies across the family of 50.2% (D), 83.5% (A) and 95.3% (E). The E-optimal design is within five points of optimal on every member of the family, and the D-optimal design is at half on one of them.

That makes the recommendation sharper than “it would not be the one that is chosen by default”. If a letter has to be picked without knowing which summary will be reported, E is nearly free insurance and D is a bet, and A — the natural compromise — is a worse maximin than E while being worse than D on D. The compromise position is dominated on the criterion it was chosen to hedge.

The deformation is corners to centre, with the edges fixed #

The three measures share nine settings and disagree about the weights, and totalling by kind says what the dial is actually moving.

At the D end the four corners carry 58.3% of the experiment, the four edge midpoints 32.1% and the centre 9.6%. At p = 64 the corners carry 21.0%, the edge midpoints 40.1% and the centre 38.9%.

So the corners fall by a factor of 2.8 and the centre rises by a factor of 4.0, while the edge midpoints move by a quarter. The whole deformation is corners into centre with the edges as a pivot, which is a cleaner statement than nine weights and says immediately what E is buying: the centre is the only setting that separates the intercept from the two squared terms, and E is spending four times as much of the experiment there.

The gap’s collapse has a rate too, and it says why the family’s endpoint is a limit rather than a member. The gap runs 0.06105, 0.03035 and 0.00115 at p → 0, 1 and 64, so between p = 1 and p = 64 it falls by a factor of 26 for a sixty-fourfold increase in p — a rate of about p^(−0.8).

Extrapolating, a gap of a millionth would need p ≈ 500,000. That is the honest reason the equivalence gap at p = 64 is a thousandth where every differentiable member reaches 10⁻¹¹: E is not merely hard to reach, it is reached at a rate that makes machine precision unattainable at any p a computation would use.

What a letter is being asked to stand for #

It is worth being explicit about what an experimenter is choosing when they choose a letter, because the family makes it sayable in one sentence and the letters do not.

Every criterion here is a summary of six variances into one number, and the only difference between them is how much a single large variance is allowed to dominate the summary. D lets it be compensated: a product of six numbers barely notices one of them being three times too big if the others are slightly better. A splits the difference. E refuses compensation entirely — one bad direction is the whole score.

Put that way, the choice is the same choice that runs through the multiple-comparison field: a familywise rate refuses to let one false positive be compensated by many true ones, and a false discovery rate allows exactly that compensation. Both are called correcting for multiplicity and they are different promises. Here both are called optimal design and they are different promises, and in this field there is not even a convention of stating which one is being kept.

The parallel is worth one more step, because it says which letter to want. A familywise rate is the right promise when a single false claim is the failure that matters, and a discovery rate is right when the claims are a batch to be triaged. E is the right criterion when a single badly-estimated combination of parameters would sink the experiment — a contrast somebody is going to test, a direction the process is known to be sensitive in — and D is right when the six parameters are a description whose overall sharpness is what is wanted. Almost nobody in either field states which situation they are in.

What is being claimed here, and what is not #

This essay claims the Φₚ family as a single object, its two named endpoints, the collapse of the eigenvalue gap that makes one of them non-differentiable, and the cross-efficiencies between them on one model and one region.

What stays out: the theory of when an E-optimal design is unique, which turns on the multiplicity of the minimal eigenvalue and is a question about the model rather than about designs; criteria outside the family, of which there are several with their own literatures; and any claim about conditioning. Eigenvalues are computed here and none of it is a statement about numerical method — that boundary belongs to numerical-linear-algebra.com and this field stays inside it, as the response-surface field did before it.

The checks #

Three claims are gated in this field’s library.

The eigendecomposition reconstructs its own matrix, entry by entry, at machine precision. Every E and Φₚ number on this page is computed through it, so this is the check that makes the rest mean anything, and it is gated at 10⁻¹² rather than loosely because it is arithmetic rather than a measurement.

The family contains what was already built, with Φ₁ against the optimality field’s A-optimal measure and Φ₀ against its D-optimal one, computed by implementations that share no derivative.

And the end of the family is the corner, asserted as two movements at once: the smallest eigenvalue has to rise along the dial and the gap has to at least halve, with the A-optimal design required to be less than fully efficient at p = 64 so that the family cannot pass by being trivial.