A variance that follows the response

The guess with two numbers in it moved the field’s design problem from the Michaelis–Menten curve to the compartmental model, $\eta(t) = e^{-\theta_1 t} - e^{-\theta_2 t}$: a rise governed by one rate and a decay by the other, the model a drug’s concentration in the blood follows after an oral dose. The design is for the slow rate, $\theta_1$, with the fast one a nuisance, and its criterion is Ds-optimality. What transferred from the simpler model was the structure; what did not was every closed form. The weights, the cross-efficiency and the support all became functions of the guess, and one of them ran to a boundary.

It listed what it left out, and one item was the noise. Every design in the field assumes the response is observed with the same variance at every sampling time, so the information a run contributes is the squared gradient of the curve at its setting and nothing else. With a variance $v(t)$ the information acquires a weight, $1/v(t)$, and the essay expected that weight to break the one exact statement the field had: that rescaling time and the rates together leaves the problem unchanged.

This essay puts the weight in. It turns out to matter in the direction everyone who has assayed a concentration would expect, and to break the invariance only for a kind of noise the earlier essay did not distinguish.

Two ways the noise can grow #

A real assay’s error is rarely constant. The commonest model in pharmacokinetics combines an additive part, the instrument’s floor, with a part proportional to the concentration being measured:

$$v(t) = 1 + \big(c\,\eta(t)/\eta^*\big)^2,$$

with $\eta^*$ the curve’s peak, so $c$ is how many additive standard deviations the proportional part reaches at the peak. At $c = 0$ the noise is constant and the field’s design is the answer. As $c$ grows the noise follows the response: large where the concentration is large, back to the floor where it is small.

The other is noise that grows with the clock — $v(t) = 1 + c\,t$, the variance rising with the absolute time at which the sample is drawn, the way a stored sample degrades or an instrument drifts over a working day. It does not care what the response is.

The guess is the field’s, $\theta = (0.2, 1.2)$, the horizon ten time units, and $c$ runs through 1, 3 and 10.

Where the design goes #

With constant noise the design is the field’s: 1.9% of the runs at time 0.66, early on the rise, and 98.1% at 5.74, well into the decay. The early setting is there to pin down the fast rate so that the slow one can be read; the late one is where the slow rate is best seen. The earlier essay found that early weight collapsing towards nothing as the two rates separate.

When the noise follows the response, the late setting moves later and the early weight falls further. At $c = 1$ the late setting is at 6.55 and the early weight 1.0%; at $c = 3$, 8.89 and 0.1%; at $c = 10$ the late setting is the horizon itself, 10, and the early weight is zero to the search’s precision. The design is chasing the tail of the curve, where the concentration is small, and the reason is the weight: in the tail the proportional noise has nearly vanished and only the floor is left, so a run there is far more precise than a run near the peak, and the slow rate’s whole effect is in the tail anyway.

When the noise follows the clock, the design moves the other way. At $c = 1$ the late setting comes in to 4.39 and the early weight rises to 4.2%; at $c = 10$, 4.03 and 4.4%. A late run is now an expensive run, and the design gives up some of the slow rate’s visibility to take its samples while they are still precise.

So the two kinds of noise push the late setting in opposite directions, from the same starting point, by up to four time units. A design built without asking which kind of noise the assay has is not slightly wrong under one of them; it is in the wrong half of the horizon.

What the constant-noise design is worth #

Under noise that follows the response, the constant-noise design is 96.6% efficient at $c = 1$, 66.7% at $c = 3$ and 35.7% at $c = 10$. Under noise that follows the clock it is 87.1%, 82.9% and 80.5%.

Two things stand out. The cost under the response-following noise keeps growing with $c$, because the design that is right keeps moving further into the tail and the constant-noise design stays where it was. And the cost under the clock levels off near four fifths, because the right design stops moving once the late runs are expensive enough — it cannot come in much further without losing sight of the slow rate altogether.

A combined error with a proportional part three times the floor at the peak is not an extreme assay; coefficients of variation of ten or twenty per cent on a concentration whose peak is many times the limit of quantification give values of that order. At that point a third of the experiment is being spent on the wrong assumption about the noise.

The invariance, asked again #

The field’s one exact statement was a scale invariance. Multiply both rates by a factor and divide the time axis and the horizon by it, and the curve at the rescaled settings is the same curve; so the optimal design rescales and its efficiency does not change, to nine decimals. The earlier essay expected a variance function to break that.

Under constant noise the rescaled design is 100.0000% efficient at every factor, as it must be. Under noise that follows the response it is 100.0000% too. Under noise that follows the clock it is 99.83% at half the time scale, 99.65% at twice and 98.10% at four times.

The reason is in what each variance reads. Rescaling time and the rates together leaves $\eta(t)$ unchanged at every rescaled setting — that is what the invariance is — so a variance written in terms of the response is unchanged too, and the weight $1/v$ is the same function of the rescaled problem as of the original. A variance written in terms of the clock reads $t$ itself, which the rescaling changes, so the rescaled problem has a different weight and a different optimum.

That sharpens the earlier essay’s expectation rather than refuting it. A weight function breaks the invariance exactly when the variance depends on something the rescaling moves, and an assay’s proportional error depends on the concentration, which the rescaling does not move. The invariance that made the rectangle a statement about ratios of rates survives the most common heteroskedasticity in the subject, and fails under the one that is about the laboratory rather than the drug.

Tailoring to the noise, priced on the rectangle #

The field’s real question was never the design at a guess but the design over a range of true rates, because the rates are what the experiment is for. A design built for the right noise at the guessed rates is a design that has fixed one thing and not the other, and it can be scored over the field’s rectangle — a factor of four each way around the guess in both rates — under each truth.

When the noise truly follows the response at $c = 3$, the design built for it averages 49.2% over the rectangle and the constant-noise design 47.8% — nearly equal — and their worst cases are 3.3% and 9.2%. The design that gets the noise right is the more fragile of the two.

The mechanism is the one the designs figure showed. Tailored to the proportional noise, the design puts all but a tenth of a per cent of its runs at one late setting, and a single setting cannot identify two rates; at the guess the fast rate is pinned by that tenth of a per cent, and at a corner of the rectangle where the fast rate is different, nothing pins it. The constant-noise design keeps two per cent of its runs early, which costs it a third of its efficiency at the guess and buys it a worst case three times as good.

So the question of which wrong guess is dearer has a two-part answer. At the guess, getting the noise wrong costs a third. Over the range of rates, getting the noise right and the rates wrong is worse than getting both a little wrong, because the noise-tailored design has committed further to a single point. The worst case in two directions found that robustness in one coordinate is worth nothing when the worst case lives in the other; here a correction in the noise coordinate actively costs robustness in the rate coordinates.

When the horizon becomes the answer #

At ten additive standard deviations of proportional error the design’s late setting is the horizon itself. It would go later if it could: the tail keeps getting quieter and the slow rate keeps being visible in it, and the only thing that stops the design is the end of the experiment. So under strongly proportional noise the design is no longer a statement about the curve. It is a statement about how long the experiment runs, and the efficiency of any design is a function of a horizon that was chosen for reasons — a ward round, a washout period — that have nothing to do with the rates.

That is a familiar shape in this field. The design that needs the answer found the local design’s settings determined by the very parameters the experiment is for; where the minimum is attained found the worst case of a design over a range sitting at the edge of the range. Here the optimum sits at the edge of the design space, and an experimenter who extends the horizon by a quarter would see the design follow it. The earlier essay recorded the horizon binding only where the slow rate was slowest; with noise that follows the response it binds at the guess.

The clock-driven noise has the opposite relation to the horizon. Its design is pulled in, to about four time units whatever the strength, and the horizon stops mattering: a longer experiment would add only late runs that the design does not want. So the two kinds of noise do not just move the design; they decide whether the length of the experiment is part of its design at all.

What a design that learns would have to learn #

A sequential design reads its own first stage and redesigns, and the design that stops guessing found that it recovers most of what a wrong guess of the rates costs. Under heteroskedastic noise there are two things to learn, and the first stage can supply evidence about both: the rates from the curve’s shape, and the variance function from how the replicates’ spread changes with the response. The second is not in any design this field has built. Whether a first stage spent at the guessed design carries enough information about the noise to move the second stage in the right direction — and what a design chosen from the data costs in coverage when it reads two quantities rather than one — is a measurement this essay does not make, and it is the one an assay without a validation report would need.

What the weight is #

Written out, the weighted information is $\sum_i w_i\, f(t_i)f(t_i)^\top / v(t_i)$, which is the information of weighted least squares — and weighted least squares with weights equal to the inverse variances is the efficient estimator. Which weights are the inverse variances made that argument for an estimator; here it reaches the design, which is chosen to make the efficient estimator as precise as possible, and so is chosen against the same inverse variances. A design that ignores them is a design for an estimator nobody should use.

That also says what an analyst has to know before designing. The variance function has to be stated, or at least its kind — follows the response, follows the clock, constant — because the three kinds move the design in different directions. An assay validation report usually states it, as a coefficient of variation at a set of concentrations; a design that does not read it has thrown away the one measurement of the noise anyone made before the experiment.

Designs searched, not certified #

Every design is searched by the field’s Nelder–Mead over the settings and weights of two- and three-point designs on the horizon, with the better kept and coincident settings merged; under every variance function here the best design had two points. The designs are searched rather than certified: the equivalence theorem under a weighted information is the second route the earlier essay checked its designs by, and it is not re-run here. The Ds criterion is the ratio of the weighted information’s determinant to its nuisance entry, the efficiency is that criterion against the design optimal at the same rates and noise, and nothing is simulated. The invariance check rescales by factors of a half, two and four at $c = 3$. The rectangle is the field’s, nine rate pairs on a geometric grid from a half to twice the guess in each rate.

The peak used to scale the proportional error is the curve’s maximum at each rate pair, $\eta^*$ at $t^* = \log(\theta_2/\theta_1)/(\theta_2 - \theta_1)$, so that $c$ means the same thing at every rate pair. Scaling it by a fixed concentration instead would make the response-following variance read the rates directly, and would break the invariance for a reason that is about the units rather than the noise.

Where the design moves under each noise, and what ignoring it costs #

The claims are that under a combined additive and proportional error the Ds-optimal design for the slow rate moves its late setting towards the horizon and its early weight towards zero as the proportional part grows, reaching the horizon at ten additive standard deviations at the peak; that under a variance that grows with time it moves the late setting in, to about four; that the constant-noise design is 66.7% efficient at three standard deviations of proportional error and 82.9% under the corresponding clock-driven noise; that the scale invariance holds exactly under the response-following variance and fails under the clock-driven one; and that over the rectangle the design tailored to the response-following noise has a worst case of 3.3% against the constant-noise design’s 9.2%.

Not claimed: anything about the maximin design over the rectangle under heteroskedastic noise, which is the obvious next object; anything about designs on a stated number of runs; and anything about a variance function estimated from the experiment’s own data, which would make the design adaptive in a second quantity.

Still open: protecting the rectangle when the noise is known #

The rectangle comparison scored two local designs, and both were poor — single-digit worst cases are what a two-point design at a guess always has over a range this wide. The earlier essay’s answer to that was a maximin design with a third setting, which protected the rectangle at a cost at the guess. Under noise that follows the response the third setting faces a new pressure: the tail is where every run is precise, so a maximin design may want its protective setting late rather than early. Whether the maximin design under the combined error keeps the equalisation property — its worst case attained at several corners at once — and how much of its worst case the right noise model buys back, is the measurement that would say whether knowing the assay’s error is worth more over a range than at a point.