The break a shared scale hid

A break that was looked for searched a regression’s errors for the row at which their persistence changes, by maximising a profile likelihood over every admissible position. On a law with a break from 0.95 to 0.65 at row sixty of a hundred and twenty, the search was a poor estimator: where the generality runs out found it averaging 74.6 with a spread of 12.3 rows, leaning towards the persistent side. Everything in this field since has explained that by the profile’s shape. A second break on a flat profile measured the one-break profile’s whole range at 6.78 units of log-likelihood and called the surface being maximised shallow.

The residue was the weights found something wrong with the transform every two-regime rule in the field had used. It restarts each regime and differences its rows, which removes the dependence, and never rescales them, so the two regimes’ whitened rows keep their own innovation variances — 0.0975 in the persistent regime and 0.5775 in the other, because the law holds the series’ variance at one and a persistence of 0.95 leaves little room for fresh noise. The profile likelihood the break search maximises is the same transform written as a likelihood, with one innovation variance concentrated out over the whole series. That essay ended on the obvious question: search for the break under a likelihood that can see the two scales, and see whether the profile was flat or the likelihood was.

Two likelihoods #

The shared likelihood is the field’s own. For a candidate break at row $b$ each segment’s persistence is its residuals’ lag-one coefficient, its rows are whitened by the restarting transform, and the log-likelihood is $-\tfrac n2 \log(\mathrm{SS}/n)$ plus the two segments’ Jacobians, with SS the whitened sum of squares over the whole series. One scale, $\mathrm{SS}/n$, for both regimes.

The per-regime likelihood changes one thing. Each segment gets its own scale, so the log-likelihood is a sum over segments, $\sum_s \big[-\tfrac{n_s}{2}\log(\mathrm{SS}_s/n_s) + \tfrac12\log(1 - \hat\rho_s^2)\big]$. It has one more parameter than the shared likelihood, and that parameter is the one the law has: two regimes whose innovation variances differ.

Both are maximised over the same positions, from a fifth of the series to four fifths, on the same residuals of the same widest candidate, on three hundred draws per law. The two searches differ only in the likelihood.

Where each puts the break #

With one scale, the search’s estimate averages 74.8 with a spread of 11.8 rows, and lands within twelve rows of the truth on 46.7% of draws — the field’s earlier finding, on these draws. With the break turned over, so that the persistent regime comes second, it averages 45.3, again leaning towards the persistent side, and lands within twelve rows on 49.3%.

With a scale per regime, the estimate averages 61.8 with a spread of 5.8 rows and lands within twelve rows on 94.3% of draws. Turned over, 56.7, spread 5.9, within twelve rows on 93.0%. The lean towards the persistent side is mostly gone — a row and three rows rather than fifteen — and the spread has halved.

That is the answer to the residue essay’s question. The bias the field attributed to a flat profile, and to the persistent regime’s steeper likelihood, was the shared scale’s. A likelihood that charges both regimes one innovation variance is fitting the persistent regime’s small innovations against a variance inflated by the other regime’s large ones, and it gains by moving the boundary into the persistent side, where every extra row it hands the persistent regime is a row of small residuals added to an average that was too large.

How much evidence there was #

Under the break, the shared likelihood’s search reports twice its log-likelihood gain at 8.82 on average. The per-regime search reports 37.04. Turned over, 8.92 and 36.38.

Four times the evidence, from the same residuals and the same candidate positions. The shared likelihood was reading the break only through the change in persistence — the lag-one coefficients on the two sides, 0.95 and 0.65, each estimated from about sixty rows and attenuated by the fit. The per-regime likelihood also reads the change in scale, and a sixfold change in innovation variance across sixty rows on each side is not subtle at all: it is the kind of change a plot of the whitened residuals would show by eye.

So the field’s description of this break as barely identifiable on a hundred and twenty rows was a description of its likelihood. The law changes two things at row sixty, its memory and therefore its fresh noise, and a search that can see only the first is looking at a quarter of the signal.

The two-unit set #

A search’s two-unit set — every position whose likelihood is within two units of the maximum — is the field’s interval for a change point, and the earlier essays refused to read it as a 95% interval because it did not cover like one.

With one scale it holds 13.5 positions on average and contains the true break on 37.3% of draws; turned over, 12.5 and 24.3%. With a scale per regime it holds 6.3 positions and contains the truth on 73.0%; turned over, 6.4 and 59.0%.

The set is half as wide and twice as likely to contain the truth. It is still not a 95% interval, and the earlier essays’ refusal stands: the convention comes from a quadratic likelihood in an identified parameter, and a change point is neither. But it is no longer a set that misses the truth most of the time.

What the extra parameter costs under no break #

A likelihood with more parameters reports more gain whether or not there is anything to find, and a criterion choosing whether to break has to charge for it. Under the three laws with no break every unit is manufactured.

Under the first-order autoregression the shared search manufactures 5.10 on average and the per-regime search 6.53; under the moving average, 4.85 and 6.78; under long memory, 4.65 and 6.32. The per-regime search costs between 1.4 and 1.9 units more, of which one is the second scale — a chi-square on one degree of freedom has a mean of one — and the rest is the search exploiting it. Its 95th percentiles under no break are 10.6, 12.6 and 10.7, against the shared search’s 8.5, 8.4 and 8.4.

Set against the gain under the break, the trade is lopsided. The extra parameter costs one and a half units of charge where there is nothing to find, and buys twenty-eight units of evidence where there is something. Choosing whether to break found that the decision depends on the charge and that the charge has to be measured rather than counted; under the per-regime likelihood the break’s evidence clears any charge the field has considered several times over.

What finding the break costs #

The point of locating the break is to whiten the regression at it, and the residue essay showed that the two-regime whitening needs both repairs — the law’s own recursion, which rescales each regime, and persistences fitted with the regression — to come within a standard error of the oracle when the break is known. With the break found by the per-regime search instead, the question is what is left.

Under the earlier construction, searching for the break cost 0.01926 at 3.1 paired standard errors, the regret rising from 0.04043 with the break known to 0.05970 with it found. Under the law’s recursion with the break found by the per-regime search, the regret is 0.01743, against 0.01640 with the break known and 0.01579 for the oracle: finding the break now costs 0.0010, at 0.3 paired standard errors, and the whole estimated rule is within 0.0016 of the oracle.

With the persistent regime second the corrected rule gives up 0.01894, against 0.01415 known and 0.01339 for the oracle — finding the break costs 0.0048 at 2.4 standard errors, more than with the persistent regime first, probably because the estimate leans three rows towards the persistent side rather than two, and a mislocated break hurts most when it hands the noisy regime’s rows to the quiet one. Both are a small fraction of the 0.0357 to 0.0423 the per-regime search and the recursion save over the earlier rule with an estimated break.

So the conclusion the field drew from its flat profile, and repeated in three essays, — that sharpening the break point cannot be done, because the data are what they are — is reversed. The data were enough; the likelihood was not reading them.

What the field’s other break numbers still mean #

Several later measurements in this collection were built on the shared-scale search, and it is worth being exact about which of their conclusions the correction touches.

What a search costs in parameters measured the likelihood a break search manufactures under no break and found it far above what a count of parameters says. That measurement stands as a statement about the shared search, and it has a sibling here: the per-regime search manufactures more, by about one and a half units, and its charge has to be measured in the same way. The principle the essay established — a searched parameter is charged by what it manufactures, not by counting it — is exactly what this essay’s charge figure applies.

Two searches, one sample and the charge that is not a sum priced a break searched alongside a window, and found the two searches’ charges overlapping. Their break is a break in the regression, searched with a likelihood whose errors have one scale; nothing in this essay’s correction applies to a break in the mean of a series whose noise is the same on both sides, and those measurements are about that object.

A break that is there found the two-unit set containing a real regression break on a minority of draws. That too is a break in the mean. Whether its two-unit set has the same defect depends on whether its law changes the scale at the break, and it does not; so its finding is not this one, and the refusal it carries — a two-unit set is not a 95% interval — stands under both likelihoods.

What does change is every statement about this law: that its break is barely identifiable from a hundred and twenty rows, that its profile is flat, and that the estimate leans to the persistent side because that side’s likelihood is steeper. All three were true of the shared likelihood and are false of the law.

What a reader could have seen #

The defect would have been visible without a second likelihood. Whitened at the right persistences, the persistent regime’s rows have a variance of 0.0975 and the other’s 0.5775, a ratio of nearly six; at the residual persistences the search actually uses, the ratio is smaller but still large. A plot of the whitened residuals against time would have shown a quiet first half and a noisy second half, and a test of equal variances between the halves would have rejected on most draws. The shared likelihood’s assumption was checkable from the quantity it computes on its way to the answer.

That is the general lesson the residue essay and this one share. A likelihood concentrates its nuisance parameters out, and a nuisance concentrated out is a nuisance nobody looks at again. The innovation variance was concentrated out once, for the whole series, in the first function the field wrote; the check that would have caught it is a look at what was concentrated.

Why the field did not see it #

Every break search in this field was written through the same prefix sums, and the shared scale was a choice made once, in the function that computes a segment’s contribution. It is the natural choice for a model whose regimes differ only in their dependence, and it is right for a law whose innovations have one variance whatever the persistence. This law is not that: it holds the series’ variance fixed, so a change in memory is a change in innovation scale, and that is the convention of every law in the field the break came from.

So the defect is in a premise rather than a computation. The likelihood was correct for a model nobody drew, and every check on it — the fast profile against the slow one, to ten decimals — checked the arithmetic of that model. A two-route check that shares the premise cannot find a wrong premise; what found it was the residue essay’s control, which handed the transform the truth and measured what was left.

Two likelihoods on the same residuals #

Three hundred draws per law, on the field’s own seed, with the residuals of the widest of the fifteen candidates searched for one break between rows twenty-four and ninety-six. Both likelihoods use the same lag-one coefficients and the same restarting segment sums; the per-regime likelihood concentrates a variance out of each segment rather than one out of the whole. The regret figures are two hundred draws on the residue essay’s seed, scored the same way: the forecast risk of the model each rule selects from the fifteen, against the best risk the oracle or least squares reaches. The per-regime search inside the regret computation is the same likelihood, written on the residual series directly.

A break located on 94% of draws once each regime has its own scale #

The claims are that under a break in persistence with the series’ variance held fixed, a break search maximising a likelihood with a scale per regime locates the break at 61.8 with a spread of 5.8 rows, within twelve rows on 94% of draws, where the shared-scale search averages 74.8 with a spread of 11.8 and lands within twelve rows on 47%; that it reports four times the likelihood; that its two-unit set is half as wide and contains the truth twice as often; that under no break it manufactures between 1.4 and 1.9 more units than the shared search; and that with it the two-regime whitening’s cost of finding the break falls from 0.0193 to 0.0010 with the persistent regime first and to 0.0048 with it second.

The claims are all about one law, the field’s own, whose variance is held at one through the break. That is the convention every law in these essays uses, which is why the correction reaches every essay that searched this break and none that searched a break in the mean.

Not claimed: anything about a break in persistence with the innovation variance held fixed instead, where the shared scale is the right model and the per-regime likelihood pays its extra parameter for nothing; anything about a search over two breaks under the new likelihood, which is the earlier essay’s pair search with a scale added; and anything about breaks in the scale alone.

Still open: a break the law hides in the scale #

The law here changes its memory and, because its variance is held fixed, its fresh noise with it, and the per-regime likelihood found the break mostly through the second. A law that changes its persistence and holds its innovation variance fixed instead — so that the series’ own variance jumps at the break — moves the evidence back into the dependence and the marginal scale, and a law that changes only its innovation variance has nothing in the dependence at all. Running both searches on those two laws would say how much of the shared search’s failure was the wrong scale and how much was this law’s particular way of tying memory to noise — the measurement that would turn this essay’s result from a correction about one law into a rule about which likelihood a break search should use.