Normal Distribution
nothing is called 95% until it has been counted
Field

The criterion, and what it assumes

D-optimality gave the catalogue back, which was reassuring, and left two things unsaid. A, D and E are one family and the letter is a choice — the design that wins the first is nearly worst at the last. And for a model whose information depends on its own parameters, a design is optimal only at a guess about the answer, which makes the cost of guessing wrong a quantity with a closed form.
The end of the family is the member the algorithm cannot reach. Every member of the Φₚ family optimised on the same 11² candidates, and two eigenvalues of each answer. The upper curve is the smallest eigenvalue of the information matrix — the quantity E-optimality maximises — which rises from 0.0993 at the D end to 0.1994 at p = 64. The lower curve is the gap between that eigenvalue and the next one up, which falls from 0.0611 to 0.0011. A smallest eigenvalue is not differentiable where it is repeated, and the family is driving the gap to zero: the one criterion here whose meaning fits in a sentence is the one whose optimum sits on a corner of its own surface. The candidate grid is on a slider and it answers a narrower question than it looks. On this square region, refining an odd grid from seven to eleven moves nothing at all — the optimum's support is the corners, the edge midpoints and the centre, and every odd grid from five up contains all of them. An even grid has no centre point and cannot reach the answer at any member of the family. On a disc, where the boundary passes through no grid point, refinement does move it.

The family behind the letters

A, D and E are not three ideas. They are three points of one family with a single dial, and running the dial from one end to the other doubles the smallest eigenvalue of the information matrix while closing the gap above it fifty-three-fold — which is the family driving its own last member to the place where it stops being differentiable.

 Where a Ds-optimal design puts its runs. The Ds-optimal measure over 121 candidate settings on a square region. It keeps 9 of them and discards the rest, and the 9 it keeps are the settings a catalogue would have offered without any of this arithmetic. What the search adds is the weights: 0.2500, 0.1250, 0.0625, which nine equal runs cannot express.

The two terms anybody wanted

D-optimality estimates all six parameters of a quadratic as precisely as possible. Nobody wants that. An experimenter looking for a maximum wants the two curvature terms, and the design that gives them is not the D-optimal one — it is a quarter of the runs at the centre, exactly, and the D-optimal design is 75.3% efficient for the question that was actually asked.

 Where to look depends on the answer. The information a single run at time t carries about the rate of an exponential decay, (∂η/∂θ)² = t²·exp(−2θt), at three values of θ. Each curve has one maximum and it is at t = 1/θ exactly — marked, and found by a search over 8,001 settings that was never told the formula. Nothing in a linear model behaves this way: there the information matrix is X′X and the parameters are not in it, so a design can be chosen once and used whatever the answer turns out to be. Here the design is optimal at a guess, and the three curves are three different experiments for one model.

The design that needs the answer

Every design this site has computed is optimal whatever the experiment turns out to say, because X′X does not contain the parameters. For a non-linear model it does, so the best place to take a measurement is a function of the number the measurement exists to find — and guessing it three times too low costs two and a half times more than guessing it three times too high.

 A design that is right once, against one that is never wrong by much. Two designs for the same two-parameter model, scored at every true value of K across a range of 16-fold. The local design is the two-point optimum for a guess of K = 1: it reaches 100% there and 66.7% at the worst point of the range. The hedged design maximises the average of log|M| over a prior spanning a factor of 4 either side, uses 3 settings rather than two, and is never below 75.4%. What it costs is 10.4 points at the one value the local design was built for — which is the whole trade, and it is only available to somebody willing to say how wrong the guess might be.

The design that hedges

A locally optimal design is right at one value of the unknown and 23.9% efficient at the edge of a sixteenfold range. Averaging the criterion over a prior instead buys the worst case back to 56.3% — and buys it by adding support points, at spreads the arithmetic decides rather than the experimenter — a third setting at a factor of 3.36 and a fourth at 8.86.

 What the main effects cost the rest of the model. The compound optimum of α·log Ds(main effects) + (1 − α)·log D on an 11-by-11 grid of the square, at 14 weights from α = 0 to 0.999, placed by its D-efficiency against the D-optimal design and its main-effect efficiency against the factorial. The D-optimal design (α = 0) gives the main effects 74.3% of what the factorial gives. At α = 0.5 the design keeps 94.6% of D and gives 85.6%; at 0.9, 64.9% and 96.8%; at 0.99, 31.3% and 99.7%. The factorial itself (open, at the left edge) gives the main effects everything and the model nothing: its D-efficiency is zero. The design optimal for the two quadratic terms (red) gives the main effects 50.0% at a D-efficiency of 83.6%.

A subset the model cannot use

Nominate the two main effects of a quadratic response surface as the parameters of interest and the optimal design is the 2² factorial — four corners, a quarter each — which leaves the full model's information matrix at rank four of six and the curvature with nothing at all. That is why the field's algorithm would not converge on it. Approached through a compound criterion instead, the trade is explicit: 96.8% of the main effects' best information costs the model 35% of its D-efficiency, and the last three points cost nine tenths of what is left about the curvature.

 What a stated power leaves of the main effects. The main-effect efficiency of the best design reaching a power of 0.8 at level 0.05, as N|δ|² rises through 50, 60, 70, 80, 100, 130, 170, 220, 300, 400, 600, 1000. a saddle, two-degree test, the compound path: out of reach, out of reach, 0.806, 0.841, 0.881, 0.914, 0.937, 0.952, 0.966, 0.975, 0.984, 0.990. a bowl, two-degree test, the compound path: 0.782, 0.823, 0.851, 0.871, 0.898, 0.923, 0.941, 0.955, 0.967, 0.976, 0.984, 0.990. a bowl, its own one-degree contrast, the compound path: 0.827, 0.859, 0.880, 0.896, 0.918, 0.937, 0.952, 0.964, 0.973, 0.980, 0.987, 0.992. a bowl, a factorial with centre points: 0.914, 0.930, 0.940, 0.948, 0.959, 0.969, 0.976, 0.982, 0.987, 0.990, 0.993, 0.996.

A design chosen for a test

A screening experiment that wants its main effects and a check on curvature has been told to trade them with a weight nobody can state. It can state a power. Read against a power of 80%, the frontier collapses to one curve in runs times the squared size of the curvature, and a large enough trial pays almost nothing. But the curve belongs to a test. A factorial with a few centre points beats the entire frontier on a bowl, spending 4% of its main-effect information where the frontier spends 10%, and against a saddle of the same size it has exactly its 5% level.

All essays