A subset the model cannot use

An optimal design is a choice of where to run an experiment, made by maximising some summary of the information the runs will carry. D-optimality maximises the determinant of the information matrix for every parameter of the model at once. The two terms anybody wanted argued that nobody wants that for a quadratic response surface, and measured what somebody might want instead: Ds-optimality for a nominated subset — there, the two pure quadratic terms that say where a maximum is — with the other four parameters treated as nuisance. Its optimum has an exact equivalence theorem and a support that factorises, and it is a design the whole model can still be fitted on.

That essay named two other subsets and measured neither. Nominating a single parameter gives c-optimality, whose designs can be supported on as few as two settings. Nominating the main effects gives the screening problem, where the answer is close to a two-level factorial and the quadratic terms become nuisance. An attempt to compute the second ran the field’s multiplicative algorithm on it and found that the algorithm would not converge. This essay finds out why, and it turns out to be the answer rather than an obstacle to it.

The optimum, read off without an algorithm #

The model is the field’s second-order surface in two factors on the square $[-1, 1]^2$: an intercept, the two main effects $x_1$ and $x_2$, their product, and the two squares. Treat the main effects as the parameters of interest. The information a design carries about them, once the other four parameters are allowed for, is the Schur complement of the nuisance block in the information matrix: $M_{11} - M_{12}M_{22}^{-}M_{21}$. For each main effect separately that is at most the variance of the factor across the design, and on $[-1, 1]$ a variance is at most one, reached only when every run sits at $\pm 1$.

So the best any design can do for both main effects is information one on each, and the only designs that reach it put every run at a corner, half at each level of each factor, the two factors orthogonal: the 2² factorial, a quarter of the runs on each corner. Its main-effect information is exactly one, the largest the square allows. It is the Ds-optimal design for the main effects, and the proof is two lines.

It is also a design on which the model cannot be fitted, which the design that cannot see a curve established for the factorial in its own right: at every corner $x_1^2 = x_2^2 = 1$, so the two squared columns are the intercept column, and the full information matrix has rank four of six. The curvature carries no information at all. That is not a numerical accident; it is the reason the multiplicative algorithm would not converge. Its update multiplies each run’s weight by a directional derivative computed from the inverse of the information matrix, and the optimum it was being asked to reach is a design whose information matrix has no inverse.

What the subset criterion needs at the optimum is only a generalized inverse of the nuisance block — the Schur complement is defined whatever the block’s rank — so the factorial can be scored on the question it answers, and here it is scored that way. It answers the main-effects question perfectly and every other question not at all.

The single-parameter case is the same fact with one factor. The information about $x_1$ alone is at most one, reached by any design whose runs all sit at $x_1 = \pm 1$, half at each — including a design of two runs, one at each end of the $x_1$ axis with $x_2$ anywhere. That is the c-optimal design the earlier essay mentioned, and it is not unique: every such design is optimal, two-point designs among them, and none of them can estimate the model.

The trade, made explicit #

An experimenter who nominates the main effects does not, usually, mean that the curvature does not matter. They mean the main effects matter more. The honest version of that is a compound criterion: maximise $\alpha \cdot \tfrac{1}{2}\log\det C_{\text{main}} + (1 - \alpha)\cdot\tfrac{1}{6}\log\det M$, which at $\alpha = 0$ is D-optimality and as $\alpha \to 1$ approaches the factorial. Every $\alpha < 1$ keeps the full information matrix nonsingular, so the algorithm’s inverse exists at every step, and the compound criterion has its own equivalence theorem: at the optimum the largest directional derivative over the candidate points is exactly one. Every point below is gated on it to $10^{-9}$, on the field’s 11-by-11 grid of candidates.

The frontier is the whole argument in one curve. The D-optimal design gives the main effects 74.3% of what the factorial gives them. Half the weight on the main effects, $\alpha = 0.5$, buys them 85.6% and keeps 94.6% of D. At $\alpha = 0.9$ the main effects get 96.8% and the model keeps 64.9%. At 0.99, 99.7% against 31.3%. The curve is flat at the top and steep at the right: the first eleven points of main-effect efficiency are cheap, costing about five points of D, and the last three, from 96.8% to 99.7%, cost the model a third of its efficiency.

The design optimal for the curvature sits well inside the curve on the other side: it gives the main effects exactly 50.0% and the model 83.6%. Nominating the curvature and nominating the main effects pull the design in opposite directions, and the D-optimal design is the compromise between them that nobody nominated.

Where the weight goes #

The compound optimum lives on nine points of the grid at every weight, eight once the centre has gone — the corners, the edge midpoints and the centre, which are the support of the D-optimal design — and the weight moves between them in a fixed order. The D-optimal design puts 58.3% on the corners. At $\alpha = 0.5$ the corners have 74.3%, the edges 22.6% and the centre 3.1%; at 0.9, 93.7%, 6.1% and 0.2%; at 0.99, 99.3%, 0.7% and nothing.

The centre goes first, and the reason is the model. A centre run has $x_1 = x_2 = 0$, so it carries no information about the main effects at all; its only use is to separate the intercept from the squared terms, which is curvature. Edge midpoints carry information about one main effect each and keep the squares estimable — a run at $(1, 0)$ is a run where $x_1^2 = 1$ and $x_2^2 = 0$ — so they are the last thing holding the full model together, and they go to zero more slowly than the centre. The corners are the factorial and take everything else.

Centre points, or edges #

The design that cannot see a curve recorded the standard repair for a factorial: add a few runs at the centre, which buy back one number about the curvature. The compound path takes the opposite route, and the difference is worth measuring because it is a difference a protocol actually makes.

A factorial with a share $c$ of its runs moved to the centre gives the main effects exactly $1 - c$ of their best information — 95% at a twentieth, 90% at a tenth — because a centre run has no main-effect information at all. Its full information matrix has rank five of six whatever $c$ is: on corners and a centre, $x_1^2$ and $x_2^2$ are still the same column, so the design can estimate their sum and not the two of them. The information it carries about the two quadratic terms as a pair is zero at every share.

The compound optimum at $\alpha = 0.9$ gives the main effects 96.8% — more than the factorial with a twentieth at the centre — while keeping both squared terms estimable, with information 0.0312 about the pair. It does that by putting 6.1% of its runs on the edge midpoints and 0.2% at the centre. An edge midpoint costs main-effect information too, but only half as much as a centre run, since it sits at $\pm 1$ on one factor, and it separates the two squares, since it sits at one on one square and zero on the other. So for the trade a screening experiment is making, a share of runs moved to the edges dominates the same share moved to the centre: more main-effect information, and both curvature terms instead of their sum.

That is the central composite design’s reasoning arrived at from the other end. A central composite puts axial runs on the axes so the squares can be told apart; the compound optimum puts its non-corner runs exactly there, at the edge midpoints, and moves them off the centre first. What the frontier adds is the price of each run in main-effect information, which is the number a screening protocol has to weigh it against.

Why the algorithm stopped #

The original failure can now be measured rather than described. At $\alpha = 0.1$ the multiplicative algorithm meets the equivalence theorem in 561 steps; at 0.9 in 939; at 0.98 in 3,593; at 0.99 in 8,800. At 0.995 twenty thousand steps leave the largest derivative $6 \times 10^{-8}$ above its bound, and at 0.999 they leave it $5 \times 10^{-4}$ above. The edge midpoints’ weights shrink roughly in proportion to $1 - \alpha$, the information about the squares shrinks with them, and a multiplicative step reduces a small weight by a factor close to one each time — so the last thousandth of weight takes longer to remove than all the rest did.

A run at $\alpha = 1$, the main effects’ own criterion, is the limit of that: a target at which the derivative is undefined, approached by a step whose size shrinks to nothing. The algorithm was not failing to find the optimum. It was converging, ever more slowly, to a design it could not represent, and the answer it would have reached in the limit is the one the argument above reads off directly.

What the curvature is left with #

The last measurement is the one an experimenter should read before nominating a subset. The information the design carries about the two quadratic terms — the curvature’s own Ds value — is 0.1883 at the D-optimal design and 0.2500 at the design optimal for it. Along the compound path it falls to 0.1227 at $\alpha = 0.5$, 0.0312 at 0.9 and 0.0033 at 0.99. From 96.8% of the main effects’ information to 99.7%, the curvature loses nine tenths of what it had.

So a screening design that is nearly optimal for the main effects is nearly useless for testing whether a quadratic term is needed, and the last three points of main-effect efficiency are where that happens. A protocol that wants both — screen, then check for curvature with the same runs — has a frontier to choose a point on, and the factorial is the one point on it where the second question cannot be asked.

What nominating a subset commits an experiment to #

A response-surface study usually runs in two stages, and the frontier is a statement about the first. A screening stage estimates the main effects in order to point somewhere — walking up the gradient is the classic use, where the fitted main effects set the direction of the next runs — and a second stage near the optimum estimates the curvature, whose sign decides whether the fitted point is a maximum, a minimum or a saddle. Nominating the main effects for the first stage is a sensible thing to do. What the measurement adds is that nominating them exactly produces a first stage that can say nothing at all about whether a second stage is needed.

The compound weight is the experimenter’s statement of how much the first stage should also be able to say. At $\alpha = 0.9$ the first stage keeps nine tenths of the curvature’s information that the D-optimal design would have had about the main effects, loses three tenths of a point of D-efficiency for each point of main-effect efficiency it gains, and can still tell, if weakly, whether the surface bends. At $\alpha = 0.99$ it can barely tell. Neither is wrong; they are different experiments, and the factorial is a third experiment that has answered the second question in advance by declining to ask it.

How the numbers were made #

Every design is a measure on the field’s 11-by-11 grid of candidate points on the square. Subset information is the Schur complement of the nuisance block computed with a pseudo-inverse — eigenvalues below $10^{-10}$ of the largest treated as zero — so it is defined on singular designs; on nonsingular ones it is the ordinary Schur complement. Efficiencies are against the factorial for the main effects, whose value of one is the bound, and against the field’s own D-optimal measure for the model. The compound optimum is found by the multiplicative algorithm on the compound derivative, started from the uniform measure and stopped when the largest directional derivative is within $10^{-9}$ of one or after twenty thousand steps.

Checked on the measurement: the factorial gives the main effects information exactly one, leaves the model rank four of six and gives the curvature nothing; along the compound path the main-effect efficiency never falls and the D-efficiency never rises, the first passing 97% near the limit; and every compound optimum to $\alpha = 0.99$ meets its equivalence theorem while at 0.999 the algorithm runs out of steps first. The refusal is the claim the quadratic terms’ case invites — that the main effects’ optimum is likewise a regular design the model can be fitted on — and the factorial’s rank breaks it.

Still open: a design chosen for a test, not an estimate #

The compound criterion trades information, which is the currency of estimation. A screening experiment is usually followed by a test — is curvature present? — and a test’s currency is power against a stated alternative, which depends on the curvature’s information through a noncentrality rather than a determinant. The point on the frontier that maximises the main effects’ information subject to a stated power for a curvature test of a stated size is computable from the measures here and a noncentral distribution, and it would replace the weight $\alpha$, which nobody can state, with a power, which a protocol can. It has not been computed.

What is claimed here, and what is not #

This essay takes Ds-optimality for the main effects of the second-order model on the square. The claims are that the optimum is the 2² factorial, with main-effect information exactly one and the full model’s information at rank four of six, the curvature carrying none; that the single-parameter optimum is any design with every run at $\pm 1$ on that factor, two-point designs included; that along the compound path the main-effect efficiency rises from 74.3% at D-optimality to 85.6%, 96.8% and 99.7% at weights of 0.5, 0.9 and 0.99 while the D-efficiency falls to 94.6%, 64.9% and 31.3%; that the weight leaves the centre first and the edge midpoints second; that the multiplicative algorithm needs 561 steps at 0.1 and 8,800 at 0.99 and does not converge in twenty thousand at 0.995 or 0.999; and that the curvature’s information falls from 0.1883 to 0.0033 along the same path.

What stays out, named as a decision: other regions, on the disc in particular, where no design puts every run at $\pm 1$ on both factors; other models, with interactions of higher order; and the test-based criterion, deferred above.

The boundary against the two terms anybody wanted is that it found the curvature’s subset optimum regular and this essay finds the main effects’ singular. Against the family behind the letters, whose criteria are all functions of the whole information matrix, the compound criterion here mixes one of them with a criterion that sees only part of it, and the frontier between the two is a statement about what an experiment is for — the point the optimum is a ratio made for a quantity that is not a parameter at all.