The residue was the weights

Where the generality runs out put a break into the errors of a regression: a persistence of 0.95 for the first sixty rows and 0.65 for the next sixty. No stationary estimate of the covariance can represent that, however many lags it reads, and every stationary rule stopped at about a third of what least squares gives up. The repair was one restart — whiten each segment as its own first-order autoregression — and told where the break was, it recovered 76.1%, giving up 0.04043 of selection regret against the best model available.

Being told the whole covariance gives up 0.01579, so the two-regime rule still gave up 0.02465, at 5.5 paired standard errors. The essay named two things in that residue. Each segment’s persistence was read off least-squares residuals, which are less persistent than the errors were, and attenuation at 0.95 is far dearer than at 0.65. And the transform restarted at the break, dropping the one link between the regimes — one of a hundred and nineteen, which it called much too small to be the residue. So it concluded the residue was the price of estimating two numbers, and named the repair: fit each segment’s autoregression jointly with the regression, so the attenuation a fit introduces is not there to be inherited.

This essay runs that repair, and then the measurement it should have been compared with first. The repair works on the estimate and barely on the regret. The residue turns out to have been something else, and it is removable entirely.

The repair that was named #

The joint fit maximises the Gaussian likelihood of the widest candidate over the two persistences at once, with the coefficients profiled out by generalised least squares at every trial pair and the transform’s Jacobian included. It is the field’s own construction, unchanged except that the persistences are chosen by the likelihood of the whole regression rather than read off residuals afterwards.

It does what it was supposed to do to the estimate. Read off residuals, the persistent regime’s persistence averages 0.812 where the truth is 0.95; fitted with the line, 0.889. The second regime’s, at 0.65, is 0.581 and 0.589: much less attenuated in the first place, as the essay predicted, and much less moved.

And it does little to the regret. The jointly fitted whitening gives up 0.03466, against the residual version’s 0.04043: a gain of 0.0058 at 1.0 paired standard errors, about a fifth of the residue.

The measurement that should have come first #

A residue attributed to estimating two numbers has an obvious control: do not estimate them. Hand the same restarting whitening the true persistences, 0.95 and 0.65, and see what is left.

Told the truth, the restarting whitening gives up 0.03488. Against the oracle that is 0.01910 at 4.9 paired standard errors — more than three quarters of the residue, with nothing estimated at all. The joint fit’s 0.03466 is the told version’s to within a fiftieth: fitting the persistences with the line recovers everything that estimating them cost. It is just that estimating them was never most of the cost.

So the earlier essay’s accounting has to be reversed. Of the 0.02465 it put down to estimation, 0.0056 was estimation and 0.0191 was the transform.

What the transform got wrong #

There are two differences between the restarting whitening and the law’s own covariance, and the earlier essay named only one of them.

The one it named is the restart. The law runs a single recursion through the break — each error is the persistence of its own regime times the error before, plus fresh noise — so the first row of the second regime depends on the last row of the first. The restarting transform treats that row as though the second regime began from its own stationary distribution. That is one link, and the essay was right that one link of a hundred and nineteen is small.

The one it did not name is the scale. The law keeps every error’s variance at one by giving each regime innovations of variance $1 - \rho^2$: 0.0975 in the persistent regime and 0.5775 in the other, a factor of six apart. Whitening a row by differencing, $y_t - \rho y_{t-1}$, removes the dependence and leaves that innovation, so the restarting transform’s whitened rows have two different variances. Least squares on them weights every row equally, which is to say it weights the persistent regime’s rows six times too heavily relative to what generalised least squares would. That is not a link; it is sixty rows, and it is what the residue was.

The law’s own recursion repairs both at once: divide each whitened row by its own regime’s innovation scale, $\sqrt{1 - \rho_t^2}$, and continue across the break instead of restarting. Told the true persistences, that transform is the inverse of the law’s Cholesky factor row for row, and its regret is the oracle’s to sixteen decimal places — 0.01579 both. That is the check that the account above is complete: nothing else separated the two-regime whitening from the truth.

Both repairs, and each alone #

The two repairs are not separable, and the figure shows it. The law’s recursion at the residual persistences gives up 0.03856 — barely better than the restarting version — because the rescaling divides each row by $\sqrt{1 - \rho^2}$, and at a persistence of 0.95 that divisor is steep: an estimate of 0.81 where the truth is 0.95 gives the persistent regime’s rows a weight of 2.9 where the right weight is 10.3, under a third of it. The correct weights need the correct persistence.

The joint fit without the rescaling gives up 0.03466, as above. With the rescaling — the law’s recursion, with both persistences chosen by the likelihood of the regression under that recursion — it gives up 0.01640, within 0.0006 of the oracle at half a standard error. That is 97.5% of the residue recovered.

And the persistent regime’s persistence is now estimated at 0.945, with a spread across draws of 0.019, against a truth of 0.95. The likelihood under the right transform pins the persistence down far more sharply than either the residuals or the likelihood under the wrong transform could, because it is the first of the three in which a wrong persistence makes the weights wrong as well as the differencing.

Under the break turned over the pattern is the same: the earlier rule 0.04040, the restarting one told the truth 0.03529, the law’s recursion at residual persistences 0.03603, fitted with the line 0.01415, the oracle 0.01339. The direction of the break changes nothing about which repairs are needed.

Why a link and a scale were easy to confuse #

The earlier essay’s argument about the link was correct as far as it went. In the precision form of a first-order autoregression — the matrix whose inverse is the covariance — the dependence across the break is a single entry, and dropping one entry of a banded matrix with a hundred and nineteen off-diagonal entries does very little. What the argument assumed, without saying, was that the diagonal of the precision matrix was right. It was not: the restarting transform’s implied precision has a diagonal of one in every row of both regimes, and the law’s has a diagonal that differs between them by the factor of six. The approximation was not in the link. It was in every row.

Iterating is not maximising found the same shape of mistake one field over: an iterated two-step rule that converged to a fixed point which was not the maximum, because the residual-based update had no Jacobian term in it. Here the rule was missing the term that makes a persistence change the weights, and without it no estimate of the persistence, however good, could have closed the gap.

Why no stationary check could have caught it #

The restarting transform passed every test the field ran on it, and it is worth seeing why, because the reason says where else to look.

Within one regime the transform is exactly right. Its first row is scaled by $\sqrt{1 - \rho^2}$ and every later row is differenced, so every whitened row of the regime has the same variance, $1 - \rho^2$ times the error variance. A common scale does not change least squares at all, so on any law with one persistence the transform is generalised least squares, and its first row — the Prais–Winsten scaling — is what makes it exact rather than nearly so. On the first-order autoregression of the field’s own comparison, the one stationary law with a single persistence to be told, the transform given the truth is the oracle row for row, and nothing about a stationary law can make it otherwise.

The defect lives only in the comparison between regimes, and only a law with two regimes has two. So the one law that could reveal it is the one law on which nothing else in the field was well estimated either — the break point searched for by a flat profile, the persistences attenuated by a fit — and a residue there had several plausible owners. The essay picked the one that every earlier field had trained it to look for, the attenuation that the fit that takes the memory out measured, and the measurement that would have exonerated it — the same transform told the truth — was one line away.

That is the habit two routes to every number exists to enforce, applied to a residue rather than a number: a decomposition is not finished until each of its parts has been removed on its own and the remainder measured.

What the weights are, in the language of a variance #

The repair has a reading that needs no autoregression in it. After whitening, each regime’s rows are independent with variance $1 - \rho_s^2$, so the regression on them is a regression with two error variances, and the efficient estimator weights each row by the inverse of its variance. Which weights are the inverse variances made that argument for blocks of a design; here the blocks are the two regimes, and the variances differ by six.

So the restarting whitening was a generalised-least-squares transform for the dependence and an ordinary-least-squares fit for the scale. It removed the correlation and kept the heteroskedasticity, and a heteroskedasticity of six to one between two halves of a sample is not small: it is the kind a practitioner would test for and correct if they saw it in raw residuals. They would not see it here, because each regime’s whitened residuals look homoskedastic on their own, and only their comparison shows the difference.

That also says how the repair generalises. A break in persistence implies a break in innovation variance whenever the marginal variance is held fixed, as it is here — and in any series whose overall variability is stable while its memory changes. A break in the marginal variance with the persistence held fixed would need the same rescaling for a different reason. Either way the transform has to divide each row by its own innovation scale, which the law’s recursion does and the restart does not.

What it changes about the break as a search #

A break that was looked for charged the two-regime whitening for searching for its change point, and found the charge per search rather than per model. Every number there is a criterion read off whitened residuals, and every whitening there is the restarting, unscaled one. The charge it measured is a property of searching a profile, and a profile computed under the wrong weights is a different surface from one computed under the right ones. Whether the charge moves when the weights are corrected is an open question; whether the selection that charge was protecting is worth more once the residue is gone is not, since the residue was most of what the two-regime rule gave up.

What the field’s other numbers inherit #

The earlier essay’s comparison between the rules still stands in its ordering: a stationary estimate of a covariance that changes is general in the wrong direction, and allowing the covariance to change once is worth more than any number of lags. What changes is the size of the gap between the two-regime rule and the truth, which was never mostly estimation, and the estimated break point.

Locating the break cost the earlier rule 0.01926 at 3.1 paired standard errors. The break point is estimated by a profile likelihood over positions computed with the same restarting, unscaled segments, so it is searching a likelihood surface that is wrong in the same way, and with the right transform it might be flatter or steeper. That is the most consequential number this essay does not re-measure.

A dependence fitted with the line measured what a joint fit buys under a stationary first-order law, where the recursion has one persistence and no scale to get wrong. It found the joint fit worth 0.0525 of correlation and little regret. Under a break the joint fit alone is worth even less — and the recursion that makes it worth something is the piece a stationary law never needed.

The earlier essay’s draws, reproduced to the digit #

Two hundred draws of a hundred and twenty rows, on the earlier essay’s seeds, so its rows reproduce its numbers to the digit: least squares, the oracle and the restarting whitening at residual persistences give up 0.16925, 0.01579 and 0.04043. Every rule is scored on the same draws by the same regret — the forecast risk of the model each selects from fifteen candidates, against the best risk the oracle or least squares can reach — and every comparison quoted is a paired difference with its own standard error.

The joint fits maximise the concentrated Gaussian log-likelihood of the widest candidate by coordinate ascent, golden-section on each persistence in turn from the residual estimates until neither moves by a ten-thousandth. Under the restarting transform the Jacobian is the two first rows’ $\tfrac12\log(1 - \rho^2)$; under the law’s recursion it is $-\tfrac12\log(1 - \rho_t^2)$ summed over every row after the first. The break is at row sixty and known to every rule here.

Unequal innovation variances, and a recursion that scales them #

The claims are that fitting the two persistences jointly with the regression raises the persistent regime’s estimate from 0.812 to 0.889 and recovers 0.0058 of the 0.0246 residue, which is what being told the true persistences recovers; that the restarting transform told the true persistences still gives up 0.0191 at 4.9 standard errors; that the cause is the two regimes’ unequal innovation variances, which the transform never rescaled; that the law’s recursion told the persistences is the oracle to machine precision; and that fitted with the line it estimates the persistent regime’s persistence at 0.945 and comes within 0.0006 of the oracle, under the break and under the break turned over.

Not claimed: anything about the estimated break point under the corrected transform, or about breaks in the scale rather than the persistence, which would make the rescaling the whole of the problem rather than most of it.

Still open: two breaks, under the right weights #

The break a shared scale hid took the next step: searched under a likelihood that gives each regime its own scale, the break that this field called barely identifiable is found at 61.8 with a spread of 5.8 rows, and the corrected rule with its break searched for gives up 0.0174 against the oracle’s 0.0158. What neither essay measures is a law with two changes of persistence. A second break on a flat profile found a pair search manufacturing as much likelihood as the first search and landing its second break on residue, under the old likelihood and the old transform. With three regimes there are three innovation scales to get wrong rather than two, and the recursion has two restarts to continue through. Whether the pair search under the right likelihood finds a real second break, and how much of the field’s conclusion that a second search is as dear as the first survives the correction, is the measurement that would say whether the two-regime result is a property of one break or of breaks.