The guess with two numbers in it

A design for a non-linear model is optimal at a guess about the answer, and every result this site has about such designs was measured on one model: the Michaelis–Menten curve, Vt/(K + t). That model has a property so convenient it is easy to mistake for a property of design theory. Only one of its two parameters moves the settings. V scales the whole information matrix and leaves the optimum where it was, so the design depends on K alone, the design for K with V a nuisance has weights that are exactly 1/√2 and 1 − 1/√2 at every K, V and horizon, and the D-optimal design is exactly 84.93% efficient for K everywhere.

Three closed forms, no simulation, and none of them survives a second parameter in the exponent.

A model where both parameters move the design #

$$\eta(t) = e^{-\theta_1 t} - e^{-\theta_2 t}$$

is a rise and a decay, one rate each: the response starts at zero, climbs while the second term dies away, and falls as the first takes over. It is the standard two-compartment curve, and what matters here is its gradient,

$$\frac{\partial \eta}{\partial \theta} = \left(-t\,e^{-\theta_1 t},\; t\,e^{-\theta_2 t}\right),$$

in which both parameters appear and neither is a scale. Change θ₂ and the second column changes shape, the information matrix changes, and the settings that maximise anything about it move.

The question is the same one the earlier field asks: the experimenter wants θ₁ — the slow rate, which is usually the one that means something — and has to estimate θ₂ alongside it. The criterion is therefore the Ds criterion, |M|/M₂₂, which is one over the variance of θ̂₁ when θ₂ is estimated with it.

What the two settings are for #

Before the numbers, it is worth saying what the design is doing, because the very unequal weights look like a mistake until the two settings are read as having different jobs.

At the guess the two rates are θ₁ = 0.2 and θ₂ = 1.2, so the response rises on a timescale of about 1/1.2 = 0.83 and decays on one of about 1/0.2 = 5. The design’s settings are 0.656 and 5.743 — one on each timescale, which is what a two-parameter curve requires: a setting where the fast term is still alive, and a setting where only the slow one is left.

But the experimenter wants θ₁ and not θ₂. The early runs are there to stop the nuisance from contaminating the estimate of the slow rate, and nothing more, so the criterion buys the smallest amount of them it can get away with — 1.9% of the experiment — and spends everything else where θ₁ is actually visible. That is the whole difference between the two criteria: D-optimality splits the runs equally because it wants both parameters equally, and Ds-optimality spends 98% of the experiment on one end because it wants one of them.

Three constants that are not constants #

At the guess θ = (0.2, 1.2) with a horizon of ten, the Ds-optimal design for θ₁ is two settings: t = 0.656 carrying 0.0192 of the runs, and t = 5.743 carrying the rest.

Set that beside the same object in the model one field back and three things have changed.

The weights are no longer fixed. There the early setting carries exactly 1/√2 = 0.7071 of the experiment whatever the parameters are. Here the early setting carries 0.0192 at the guess, 0.2508 at one corner of a fourfold rectangle around it, and 2.6 × 10⁻⁹ at another. The design does not merely change proportions: it changes which end of the experiment the runs are at.

What D-optimality costs is no longer a constant either. The design that treats both parameters as wanted is 84.93% efficient for K everywhere in the earlier model — a number, checked to nine decimals at five values of K and two horizons. Here the same cross-efficiency runs from 0.5000 to 0.7501 across the rectangle. “What it costs to answer the wrong question” stops being a figure to quote and becomes a picture to draw.

And the asymmetry is enormous in the other direction. A design built for θ₁ alone is between 0.0001 and 0.7726 efficient for the pair. At the far corner it is not merely worse for the pair, it is useless for it: a design that spends 2.6 × 10⁻⁹ of its runs at the setting which identifies θ₂ has essentially no information about θ₂ at all, which is exactly what “θ₂ is a nuisance” was allowed to mean.

Where the optimum stops being attained #

That last number is not a search failing. It is the criterion, and it is worth following because it is the strongest argument this field has for not optimising at a point at all.

Write the two-setting design as weights w and 1 − w on settings t₁ and t₂. Then

$$|M| = w(1-w)\big(f_1(t_1)f_2(t_2) - f_1(t_2)f_2(t_1)\big)^2, \qquad M_{22} = w f_2(t_1)^2 + (1-w) f_2(t_2)^2 .$$

Where the two rates are far apart, f₂ is concentrated near t = 1/θ₂ and is essentially zero at the late setting, so M₂₂ ≈ w f₂(t₁)². The w in the numerator and the w in the denominator cancel, the ratio increases all the way down, and the supremum is approached rather than attained. The “optimal design” for θ₁ at those parameter values asks for a vanishing share of the runs at the one setting that identifies the nuisance — which is not a design anybody can run, and which no amount of care in the search will turn into one.

The second route, when there is no closed form #

Every design in the two fields before this one could be checked against an expression. This one cannot: it is found by a derivative-free search, and a search returns wherever it stopped.

The equivalence theorem is what makes the answer checkable anyway. For Ds-optimality with one parameter of interest, a design is optimal exactly when

$$d(t, \xi) = f(t)'M^{-1}f(t) - \frac{f_2(t)^2}{M_{22}} \le 1 \quad\text{for all } t,$$

with equality at every setting the design uses. That is a statement about the whole design space, so it can be checked at two thousand settings while the design has two — and at the guess the largest value anywhere is 1.000000247, attained at t = 5.744, which is one of the design’s own points.

The check is run at four parameter points and two horizons, and it is the only thing standing between “the design is optimal” and “Nelder–Mead returned this”. It is also what says the degenerate designs above are genuinely at the supremum rather than genuinely broken.

Where the experiment runs out before the information does #

The tables above have a second pattern in them and it is not the criterion’s doing. At θ₁ = 0.1 the late setting of every design sits at exactly 10.000 — the end of the horizon — where at θ₁ = 0.2 and 0.4 it is interior, at 5.743 and 4.014.

That is the design asking for a setting the experiment does not have. A slow rate of 0.1 has a timescale of ten, so the point at which the response is most informative about it is past the last moment anybody is measuring, and the design does the only thing it can: it piles the runs on the boundary. Every quantity in this field behaves differently at those parameter points, and the reason is not statistical at all.

It matters for reading the rest of the field because it means the rectangle has two kinds of corner in it. At three of the nine parameter points the horizon binds, and there the question is “is the experiment long enough”; at the others it does not, and there the question is “is the guess about the ratio right”. A single worst-case number over the rectangle is a minimum over both kinds, and which kind it lands on is worth knowing before anybody is told that a design guarantees 46%.

Two per cent of an experiment, and why it cannot be rounded #

The weight of 0.0192 at the early setting is the smallest number in this field and it is worth reading as a quantity an experimenter has to deliver rather than as a property of a criterion.

One run in fifty-two is what it costs to stop the nuisance rate from contaminating the estimate of the slow one. Set against D-optimality’s even split, that is the whole difference between wanting both parameters and wanting one: 50% of the experiment against 1.9%, for a term that appears in the answer only as something to be removed.

What makes it awkward is that it is small and cannot be made smaller. Round 0.0192 down to zero and the design has one support point for a two-parameter model: the information matrix is singular, neither rate is estimable, and the criterion is not merely worse but undefined. The optimal weight is 1.9% and the smallest admissible weight is anything above zero, so the quantity separating a good design from an unusable one is a couple of runs.

A design measure is a share and an experiment is a whole number of runs, so the two meet at n = 52 — the smallest experiment in which one run is at most the optimal share. Below that the exact design has to place a whole run at the early setting and therefore overspend: at twenty runs, one run is 5% against an optimum of 1.92%, which is 2.6 times too much of the experiment at a setting whose only job is to identify something nobody wants. That is a rounding error of a size that has to be priced rather than ignored, and pricing it is what the design that has to be integers is for.

The general shape of it is that Ds-optimality produces extreme weights, extreme weights are exactly where the gap between a measure and a realisable design is largest, and the model with one parameter in the exponent hid this too: 1/√2 is a comfortable share that any experiment can deliver to within a run or two.

The collapse is faster than compounding #

Two worst-case numbers are quoted for the same design on two rectangles: 56.3% on a rectangle twice as wide in each coordinate and 7.3% on one four times as wide. Between them the range of ratios θ₂/θ₁ goes from 3-to-12 — a factor of four — to 1.5-to-24, a factor of sixteen. The range squares when the width doubles, because a ratio has the width in both its numerator and its denominator.

If the guarantee degraded as a power of that range, the second number would be the first one squared: 0.563² = 31.7%. It is 7.3%, which is more than four times worse. The guarantee does not compound; it accelerates, and the reason is the degeneracy in the previous section rather than any smooth loss of efficiency. Somewhere between a fourfold and a sixteenfold ratio range the criterion stops asking for a small share of runs at the early setting and starts asking for a vanishing one, and a design built at the guess has no runs to spare there when it arrives.

That is the honest form of the warning this essay ends on. A design’s efficiency over a range is not a quantity that can be extrapolated from a narrower range, because the failure it is heading for is not a gradual one. Halving the uncertainty about the two rates did not double the guarantee; it multiplied it by nearly eight.

The invariance that says what the rectangle is #

There is one closed form left, and it is the one that makes the two-dimensional problem comprehensible rather than merely large.

Multiplying both rates by c and dividing every setting by c leaves the model unchanged — it is the same curve read on a different clock — so every efficiency must be unchanged too. Checked as an identity, it is: at three parameter points and three scale factors the two agree to nine decimals.

The consequence is that the criterion values repeat exactly wherever the horizon does not bind. The design at (0.2, 1.2) and the design at (0.4, 2.4) have the same weight, 0.0192, and the same cross-efficiency, 0.5225, to every digit. What a parameter point means for this problem is its ratio and its horizon in the units of its own rate, so a rectangle “four times wide in each coordinate” is a range of ratios from 1.5 to 24 together with a range of horizons — and the corners where the horizon binds are visibly different in every table above, because there the design has run out of experiment rather than out of information.

One more consequence of the invariance is worth drawing out, because it changes what a “wide” range means. A rectangle four times wide in each coordinate is not four times of uncertainty — it is sixteen times of area, and the ratio θ₂/θ₁ inside it runs from 1.5 to 24, a factor of sixteen. An experimenter whose guess is good to a factor of two in each rate has said something much vaguer about the quantity the design actually depends on, and the arithmetic of that is not obvious from the statement.

What a guess is worth here #

The guess is a point and the truth is not. The design built at θ = (0.2, 1.2) is 100% efficient there by construction, averages 55.0% over the rectangle, and its worst corner is 7.3%.

Seven per cent is the number to carry out of this essay. It means an experiment run at that design, if the rates are at that corner, is worth about a fourteenth of the experiment somebody who knew the answer would have run — and there is no warning inside the experiment that this has happened.

The D-optimal design at the same guess is the interesting comparison. It is only 52.2% efficient for θ₁ at the guess itself — it is answering the wrong question — and its worst corner is 25.6%, three and a half times better. A design that is wrong everywhere by a fixed amount beats a design that is exactly right in one place, as soon as the place is uncertain. That is not an argument for D-optimality; it is an argument for asking what the worst case is before optimising anything, which is where this field goes next.

What is claimed here, and what is not #

This essay takes a design problem whose guess has two numbers in it, and the claim is the comparison with the model that has one: which of the earlier field’s results were about design theory and which were about Michaelis–Menten.

The answer is that the structure transfers and the numbers do not. Ds-optimality is still a ratio of determinants, the equivalence theorem still holds and is still the second route, and the design still puts most of its runs where the parameter of interest is best seen. What does not transfer is every closed form: the weights, the cross-efficiency and the support are all functions of both parameters here, and one of them runs to a boundary.

What stays out and is named as a decision: models with three or more parameters, where the subset becomes a choice as well as a criterion; heteroskedastic responses, where the information matrix acquires a weight function and the whole invariance above fails; and exact designs on a stated number of runs, which is the integer problem this site has measured elsewhere and which would confuse two effects here.

The boundary against the criterion field is the criterion. Everything about what Ds-optimality is, why the ratio of determinants is the right object and how the equivalence theorem is stated belongs there and is used here unchanged.

The checks, and what they are standing in for #

Four claims are gated in this field’s library, and they matter more than usual because there is no closed form to fall back on.

The equivalence theorem is required to hold — sensitivity at most one everywhere, and one at a support point — at four parameter points and two horizons. The scale invariance is required as an identity to nine decimals, which is the only exact statement in the whole file. The weights are required to vary by more than a factor of three across the rectangle while the D-optimal design’s stay exactly equal, which is the “constant belonged to the model” claim stated so that it would fail if it were false. And the degenerate case is required to be genuinely degenerate: the early weight below 10⁻⁵ at a ratio of 24 and above 10⁻³ at a ratio of 6, with the equivalence theorem still satisfied, so that the collapse is a property of the criterion rather than of the optimiser.

There is no refusal in this essay, and the reason is worth stating rather than leaving as an absence: the refusals for this field are about what a design is asked to guarantee, and they belong to the essay that asks it.