Where the minimum is attained

Every optimal design on this site so far has been found by climbing. The multiplicative algorithm moves weight towards wherever the criterion’s derivative says it should go, and it stops when a theorem says stopping is correct: for D-optimality, when the largest prediction variance over the candidate settings equals the number of parameters, checked at machine precision and shown to be an equality rather than a target.

The maximin design cannot be found that way, and the reason is the same one the criteria field ran into at the far end of its family. A minimum over a range is not differentiable where the minimum is attained more than once, and the optimum is exactly where it is attained more than once.

Why the optimum has to be a tie #

The argument takes one paragraph and no arithmetic.

Suppose a design’s worst efficiency over the range is attained at a single parameter value — call it the binding value. Then in a neighbourhood of the current design the objective is just the efficiency at that one value, which is a smooth function of the design, and moving the design a little towards the local optimum for it raises that efficiency and therefore the worst case. The design was therefore not maximin. So at the optimum the minimum cannot be attained at one value alone: it has to be attained at two or more, and any move that helps one of them hurts another.

A tie is what having finished looks like. That is not an analogy with the equivalence theorem — it is the same kind of statement, and it is checkable in the same way.

Over the quarter-to-four range the design’s worst case is 78.74% and it is attained at K = 0.25, K = 1.00 and K = 4.00 — the two ends and the guess. The rest of the curve sits between 78.74% and 80.8%, so the whole picture is flat to two points, which is what a solved maximin problem looks like: nothing left to gain anywhere, because everything that could be gained at one parameter value costs more at another.

The design that does it is three observation times and three weights, and both halves are worth stating because the weights are the part nobody guesses. It observes at t = 0.298, t = 2.029 and t = 9.992, with 23.4% of the runs at the first, 32.2% at the second and 44.4% at the third. The times are roughly one early, one near the guessed parameter’s natural scale and one late; the weights rise monotonically along them, which is the opposite of the intuition that says the uncertain early part of a decay curve needs the most looking at. The ordering follows from which settings are useful across the whole range rather than at any one K: the late time earns the largest share because it says something about the decay whatever K turns out to be, and the early time earns the smallest because it is informative only in the corner of the range where K is small. A design protecting a range buys the settings that never go to waste, and pays for them with the ones that pay off only under one value of the parameter.

The same design from the other end: a least favourable prior #

There is a second route, and it is worth having for the reason every second route on this site is worth having — it shares no arithmetic with the first.

A maximin problem is a minimax problem, and a minimax problem is an averaging problem under the worst possible weighting. Choose a prior on the range, find the design that maximises the average criterion under it, and score that design’s worst efficiency. Some priors produce designs with poor worst cases; the prior that produces the best worst case is the least favourable one, and the design it produces is the maximin design.

So the search can be run over priors instead of over designs, with the criteria field’s existing averaging machinery inside it: put more weight wherever the current design is weakest, recompute, repeat. Nothing in that loop knows what a maximin design is.

It arrives at a worst case of 78.63%, against the direct search’s 78.74% — eleven hundredths of a point apart, on two searches that share nothing below the efficiency function itself.

And it supplies something the direct search cannot: the prior. Its weight lands at K = 0.25 with 39.8% and K = 4.00 with 40.5%, with 7.4% at K = 2.83 and almost nothing anywhere else — and the efficiencies at those points are 78.93%, 78.63% and 80.13%. The design is equally efficient at every parameter value the least favourable prior uses, which is the equalisation property stated as a fact about two objects at once rather than as a property of one curve.

What the prior weights are, and what they are not #

The least favourable prior is a computational device and it is tempting to read it as a belief. It is not one, and the distinction is worth a paragraph because the two objects have the same type.

A Bayesian prior says what an experimenter thinks is likely. A least favourable prior says which parameter values are binding — where the design is being held back — and it is a property of the range and the model rather than of anybody’s opinion. Its support is the set of values where the design cannot be improved without giving something up, which is why the weights sit at the ends here: a design that could be improved at K = 1 without cost at K = 0.25 or 4 would already have been.

Read that way the prior is a diagnostic. Weight at an interior value — the 7.4% at K = 2.83 above — says that the constraint is not only at the ends, and a range whose least favourable prior is concentrated at one point is a range on which maximin has collapsed to local optimality.

The standardisation is not a detail #

Everything above compares efficiencies: the information a design carries at K, divided by the information the best design for K would have carried. There is an obvious-looking alternative that skips the division and maximises the worst determinant of the information matrix directly. It is the same words — protect the worst case — with a different quantity inside them.

The unstandardised search returns a two-point design at t = 2.22 and t = 10.00 with equal weights — which is exactly the locally optimal design for K = 4, the top of the range. That is not a coincidence and it is not a numerical failure. Information at one parameter value is not comparable with information at another: the determinant is smallest at K = 4 whatever any design does about it, so the minimum over the range is pinned there, and maximising it is the same problem as maximising the determinant at K = 4 alone.

A design offered as robust over a range is then the local design for one end of it, wearing a robust design’s name. Its worst efficiency is 29.0% against the standardised search’s 78.7%, and nothing in its construction or its output announces the substitution.

That failure is a general one and it is why this is the field’s refusal. Any minimax problem over a family of situations needs the situations put on a common scale first, and the scale has to be something that means the same thing in each: a ratio to the best available, not a raw quantity. The same argument appears in the exact-inference field as the requirement that a power comparison be scored where both procedures were meant to run, and in the criteria field as the reason its hedge comparison had to be rewritten.

When the tie is only two, the design is a local design in disguise #

The narrow-range picture above is worth one more sentence, because it shows the structure degrading gracefully rather than breaking.

Over a factor of two the maximin design uses two settings, ties at the two ends, and looks exactly like a locally optimal design for some interior value of K. It is one: the local design for K ≈ 0.9, which happens to be the value whose two-point optimum balances the two ends. Nothing has gone wrong — the range is narrow enough that a single well-chosen local design is the best worst-case design available, and the search discovers that rather than being told it.

The third setting appears when the range grows past the point where any single local design can hold both ends, which for this model is at about a factor of three either side. That transition is an integer changing, it is decided by the arithmetic rather than by the designer, and it is the same staircase the averaged design’s support count climbs — reached from a different problem and landing on the same step.

Why the search is a different kind of algorithm #

The direct search here is a derivative-free simplex method run over designs of two, three, four and five settings, with the best kept and coincident settings merged afterwards. That is a blunt instrument by the standards of the optimality field, and the reason it is used is the whole subject of the essay: the objective has corners exactly where the answer is.

Three consequences worth naming, because they are what makes a maximin design harder to produce than an averaged one.

The number of settings is part of the answer. For this model, restricted to two settings, the best worst case is 69.5%; allowed more, it is 78.8% with three. A search that fixed the support size at the local design’s would have returned the first and reported success.

There is no stopping theorem in the D-optimal sense. What is checked instead is the equalisation property — that the minimum is attained at more than one value, and that a second search agrees to within a tenth of a point. That is weaker than an equality at machine precision, and it is what this problem admits.

The answer moves with the range and not with the guess. Nothing in the search reads a prior, so two experimenters who agree about what is possible get the same design whatever they believe.

The parallel with E-optimality, stated exactly #

The criteria field’s Φₚ family runs from D at one end to E at the other, and E — maximise the smallest eigenvalue of the information matrix — is the member the multiplicative algorithm cannot reach, because a smallest eigenvalue is not differentiable where it is repeated and the optimum repeats it. Measured there: the smallest eigenvalue rises from 0.09927 to 0.19943 along the family while the gap above it collapses fifty-three-fold.

This field’s obstruction has the same shape with the index running over parameter values rather than over eigenvalues. E-optimality minimises over the directions in which a parameter can be estimated badly; maximin minimises over the parameter values at which the design can perform badly. Both are minima of smooth functions over an index set, both are attained at more than one index at the optimum, and both therefore have a corner where the answer is.

The difference is which index set is being minimised over, and it is not a small one: the eigenvalue version is a fact about a single information matrix, and this one is a fact about a family of them that only exists because the model is non-linear.

Six and a half points a doubling #

The tie makes the worst case a single number at each width, and the five widths measured turn that into a rate rather than five separate results.

The guarantees run 96.7%, 90.8%, 78.8%, 72.2% and 64.6% as the range grows, and the ranges they belong to double their extent each time past the first pair. Differenced per doubling of the range, the falls are 5.9, 6.0, 6.6 and 7.6 points. The guarantee costs about six and a half points for every doubling of the range being protected, and the rate is slowly accelerating rather than flattening.

Both halves of that are useful and the second is the one worth carrying. A quantity falling at a constant rate per doubling would be reassuring: it would mean protection gets cheap in the tails and a very wide range is not much worse than a wide one. It does not. Each doubling costs a little more than the last, so the sixty-fourfold range’s last doubling cost 7.6 points where the fourfold range’s cost 5.9, and extrapolating the trend puts the guarantee at about half by a range of two hundred and fifty.

That gives an experimenter the trade in the currency they set. Declaring a range wide enough to be safe is not free and its price is now quotable: from a factor of four to a factor of sixteen costs twelve points of guaranteed efficiency, and from sixteen to sixty-four another fourteen. Whether that is worth paying is the question the design that hedges asks in a different currency, and the answer depends on how badly a guess outside the range would be served — which is a quantity the maximin construction deliberately says nothing about.

The third setting, and why there is never a fourth #

The support count is part of the answer here, and the two numbers quoted for it — 69.5% restricted to two settings, 78.8% allowed more — put a price on the constraint nobody declares.

The third setting is worth 9.3 points at a sixteenfold range. Below about a ninefold range it is worth nothing, because a single well-chosen local design still balances both ends and the search finds two settings on its own. Above it, the two-setting restriction is a nine-point loss that a design report would describe in exactly the same words as the unrestricted answer.

What does not happen is a fourth. The count goes two, then three, and stops there however wide the range gets — the sixteen-to-one and sixty-four-to-one designs both use three settings, spread further apart. So past a ninefold range the entire adjustment to a wider range is redistribution of weight among three fixed roles rather than the addition of new ones, and that is why the guarantee keeps falling: the design has run out of structural responses and is left trading the ends against each other.

The ceiling is not arbitrary. A two-parameter model’s information matrix is 2×2, a design measure needs at most three support points to make any such matrix extreme, and the third is the one the tie requires. A model with more parameters would have a higher ceiling, which is the direction the two-parameter guess points and the reason its degenerate corner has nowhere to go.

What a reader should check in somebody else’s robust design #

Four questions, each of which this essay has made answerable, and none of which requires access to the search that produced the design.

What was standardised? If the answer is “the determinant” or “the variance”, the design is a local design for one end of the range and the robustness is nominal. The efficiency has to be a ratio against the best available at each parameter value.

Where is the minimum attained? Score the design across the range and look at the low points. One low point means the search stopped early; the values should tie, and the tie should include the ends of the stated range.

How many settings does it use? A design with the same support size as the local design, over a range wide enough to need more, is a design that was searched under a constraint nobody declared.

What range was assumed? It is the only assumption in the construction, it is doing all the work, and a design’s efficiency outside it is not protected by anything.

What is claimed, and what is not #

The claim is the structure of the maximin optimum: that the worst case is attained at more than one parameter value, that a search over priors reaches the same design as a search over designs, that the least favourable prior’s support is where the ties are, and that standardising the criterion across parameter values is what makes the problem the intended one.

What stays out: a proof of the minimax theorem, which is mathematics and belongs where mathematics is owned; maximin over continuous ranges treated analytically rather than on a grid; and the question of whether the least favourable prior is unique, which is real and which the measurement here cannot settle — two runs of the outer search from different starting weights agree on the worst case to a tenth of a point and are not required to agree on the prior.

The checks, and the refusal #

Four claims are gated. The worst case must be attained at more than one parameter value, with both ends of the range among them. The prior route and the design route must agree on the worst-case efficiency to within a point. The efficiencies at the parameter values the prior uses must be equal to within two and a half points, which is the equalisation stated as a number rather than as a picture. And the maximin design must use more settings than a local design does.

The refusal is the unstandardised search, against the standard that a design offered as robust over a range should be worth having at every value in it. It returns the local design for one end and is 29.0% efficient at the other, and the check requires it to fail — because a check that has never rejected anything proves nothing.