A break that is there

A charge that depends on the rule ended with two refusals it could not yet turn into measurements. The plateau widths it reported — the rows within two log-likelihood units of the profile’s top — were the profile’s two-unit sets, which the earlier field refused to read as 95% intervals; and a calibrated interval for a break point found by a joint search would need its own simulation. Both refusals were made at a null, where there is no break point and nothing for an interval to cover. That is the easiest case to refuse and the least useful one: a practitioner reaches for an interval for a break point because they believe there is one.

This essay puts one there. The question is what the two-unit set does when it has something real to contain, what threshold would make it honest, and whether searching the whitening window as well — the rule this field has been pricing — helps or hurts.

A shift from row sixty #

The regression is the field’s own: a response on four predictors, with errors from a first-order autoregression at 0.8, and 120 rows. The alternative adds a constant shift to the response from row 60 onwards, of 0.3, 0.6, 1, 1.5 or 2 standard deviations of the error. The break search places a split at every candidate row from 24 to 96 — seventy-three of them — and reads the profile likelihood of the split regression at each.

The profile is the one a break that was looked for built for a split regression, read through prefix sums so that every candidate row costs one subtraction, and the charge it manufactures under a null is the quantity what a search costs in parameters measured at about two and a half parameters. Nothing about that search changes here; only the data have a break in them.

Two versions of the search run on every draw. One is the field’s fixed window: no whitening, the errors treated as independent. The other is the joint search, which also chooses a band-whitening window from the field’s list and reads the profile through the window it chooses; at these break sizes it picks between 4.1 and 5.0 lags on average. Three hundred draws at each size.

On this sample both searches put their top at the break or a row from it, and the two profiles differ in scale rather than in shape. The unwhitened one is steep: it is nearly twenty units below its top within ten rows on either side, because it reads the autocorrelated errors as independent and so credits every row with more information than it carries. The whitened one, at the two lags the joint search chose here, rises about half as far over the same rows. Neither profile is narrow at the two-unit line: each holds a handful of rows there, and the true break is among them on this draw, which is the exception rather than the rule.

Two units contain it a fraction of the time #

At the fixed, unwhitened window the two-unit set contains row 60 on 12.0% of draws at a break of 0.3 standard deviations, 19.0% at 0.6, 31.7% at 1, 51.7% at 1.5 and 69.3% at 2. With the window searched it does better at every size — 22.7%, 30.0%, 44.3%, 63.3% and 75.3% — but at no size does it come within twenty points of 95%.

The whitened sets are wider, which is most of why they cover more: 16.3 rows on average at the smallest break against 8.0 unwhitened, and 4.9 against 3.3 at the largest. Whitening flattens the profile, so more rows sit within two units of the top.

The refusal the earlier fields made at the null therefore stands here too, with numbers where it had only a principle: a two-unit set for a break point is not an interval, whether or not there is a break, and whether or not the errors were whitened first.

What a set would need #

The direct question is how far below the profile’s top the true row sits, draw by draw. That deficit’s 95% point is the threshold a set would need in order to contain the truth on 95% of draws, and it can be read straight off the same runs.

Unwhitened, it is 19.49 log-likelihood units at a break of 0.3, 19.07 at 0.6, 20.27 at 1, 16.96 at 1.5 and 13.54 at 2 — between about seven and ten times the convention. With the window searched it is 8.33, 8.74, 8.51, 6.74 and 5.47: still between about three and four and a half times the convention, and under half the unwhitened figure at every size.

The two readings tell two different things. The ratio between them — about two and a third at the small breaks — is the profile’s overstatement of its own curvature when it treats autocorrelated errors as independent: the unwhitened likelihood is too steep by roughly that factor, so its deficits are inflated by it, and whitening removes most of the inflation. That is a calibration error in the likelihood itself, and the window search fixes much of it.

What whitening does not fix is that the needed threshold is not a constant. Even whitened it falls from about 8.5 units at small breaks to 5.5 at a break of two standard deviations, because a large break makes the profile genuinely peaked at the right place and a small one does not. A set calibrated at one break size is wrong at another, and the size of the break is the thing being estimated. There is no fixed threshold that makes the profile’s set an interval across break sizes; the best a fixed threshold can do is be honest at the hardest size and conservative elsewhere.

How wide an honest set is #

Taking the hardest size measured as the calibration — a threshold of 9 units whitened, 21 unwhitened, each just above that search’s largest needed threshold — gives a set that contains the truth on most draws at every size, and its width is the honest answer to how precisely a break point is located.

With the window searched, the 9-unit set contains row 60 on 93.0% of draws at a break of 0.3 and between 93.7% and 98.0% at the larger ones, and holds on average 66.2 rows at 0.3, 57.1 at a break of one standard deviation and 30.2 at two. Unwhitened, the 21-unit set holds 67.5, 57.7 and 31.7 rows. Out of seventy-three candidates.

The two widths agree to within a row or two at every size. That is the cleanest statement of what whitening does here: it does not locate the break more precisely, it makes the likelihood’s own scale honest, so that a threshold chosen on the usual grounds would be closer to right. The information about where the break is was the same in both profiles all along.

And the information is small. At a break of one standard deviation — a shift a practitioner would call substantial — an honest set holds four fifths of the rows a break could be placed at. The profile’s peak looks informative because a two-unit set around it is narrow; the narrowness is the convention’s, not the data’s.

Landing near it #

The point estimate tells the same story more gently. At the fixed window the search lands within five rows of row 60 on 16.7% of draws at a break of 0.3 and 76.3% at 2; with the window searched, 15.7% and 76.0%. The root-mean-square distance from the truth is 23.2 and 23.4 rows at the smallest break, 17.5 and 19.1 at a break of one standard deviation, and 9.7 for both at the largest.

So searching the window does not move where the break is found by more than a row or two of error at any size. That answers the other half of the field’s question. A charge that depends on the rule found that under a null the break point found with the window searched differs from the one found at a fixed window on three draws in four; under a real break the two land in nearly the same place, because the break is now a feature of the data that both searches find rather than luck that each finds differently. What the window search changes is the scale of the profile around that place, which is what an interval is made from.

A small break is nearly no break #

The smallest break measured is worth setting beside the null the earlier essays measured, because the comparison says how much a modest real break changes what the profile reports.

Under no break at all, the earlier field found the two-unit plateau 8.9 rows wide at the fixed window and 18.1 with the window searched. With a shift of 0.3 standard deviations from row 60 it is 8.0 and 16.3: a tenth narrower. The profile has barely noticed the break.

The point estimate says the same. If the search placed its break uniformly at random among the seventy-three candidate rows, it would land within five rows of row 60 — eleven of the seventy-three — on 15.1% of draws. At a break of 0.3 it lands there on 16.7% of draws at the fixed window and 15.7% with the window searched. At a break of 0.6 the figures are 19.3% and 15.7%. A shift of six tenths of a standard deviation, sustained over half the sample, is a break the search cannot place better than a guess.

That is not a failure of the search. Errors correlated at 0.8 carry, about a level, the information of $(1-\rho)/(1+\rho)$ of their number of independent rows — one ninth, so a hundred and twenty rows are worth about thirteen. A shift of 0.6 standard deviations estimated from two segments of six or seven effective rows each is a difference smaller than its own standard error, and the noise’s slow wandering produces boundaries between high and low stretches that look exactly like it. The search finds the most convincing such boundary, and only when the break is larger than the noise’s own wanderings is that boundary the break.

Finding it and placing it are different questions #

The field has already measured how often a break of these sizes is detected. With the threshold calibrated for what the break search adds after the window search, the charge that is not a sum found the joint test rejecting on 9.6% of draws at a shift of 0.3, 16.8% at 0.6 and 29.2% at 1, on the same law and the same break position.

Set beside the figures here, the two halves of the problem separate cleanly. At a break of one standard deviation the joint search detects the break on about three draws in ten, places it within five rows on 32.3% of draws, and an honest set for its position holds fifty-seven of the seventy-three candidate rows. Detection, location and the precision of location are three measurements, and the second and third are worse than the first suggests: a test that has fired says there is a break somewhere; it says much less about where.

A practitioner who reports a break point therefore owes two statements, not one — the evidence that a break exists, which the calibrated joint test supplies, and the evidence about its position, which the profile supplies and the two-unit set overstates. The usual report joins them into a single date with a narrow bracket, and on a sample like these both halves of that report are stronger than the data.

Why whitening changes the scale and not the place #

It is worth being precise about what the window search did, because it is easy to read the coverage figure as saying that whitening locates breaks better, and it does not.

Where the profile peaks is decided by where the shifted segment’s evidence is strongest, and the whitening — a fixed linear transformation of the rows, chosen once for the sample — moves that evidence around very little. That is why the two searches land in the same place to within a row or two of error, and why their honest sets, each at its own calibrated threshold, have the same width.

The window the joint search chooses moves with the break, slightly and in one direction: 4.90 lags on average at a break of 0.3, 4.98 at 0.6, 4.78 at 1, 4.45 at 1.5 and 4.12 at 2. A larger break, once modelled, leaves residuals with less apparent persistence, so the search wants less whitening — which is the same coupling the earlier field found running the other way at the null, where an unmodelled stretch of high values pushed the window wider. Here the effect is a fraction of a lag and does not move the location.

What the whitening changes is how many log-likelihood units the profile assigns to the same evidence. Treating rows that are correlated at 0.8 as independent credits the sample with far more information than it has, so every gap in the profile is too large, and the deficit at the truth that a 95% set has to accommodate comes out about two and a third times larger than it does once the correlation is accounted for. The whitened profile’s units are closer to the ones a two-unit convention assumes — and still, at between about three and four and a half times the convention, not close enough.

Still open: a set calibrated by simulation at the estimate #

The construction this points to has not been run. A calibrated set for a break point would simulate the profile’s deficit at the estimated break, under the estimated shift and the estimated error dependence, and use its 95% point as the threshold for that sample — a parametric bootstrap of the deficit. Its threshold would track the break size rather than being fixed at the hardest one, so its sets should be narrower at large breaks than the 9-unit sets above and no wider at small ones. Whether it holds 95% when the shift it simulates from is itself estimated with the error the figures above show — a root-mean-square of twenty rows at small breaks — is the open question, and it is the same transport problem the charge that is not a sum met when a threshold calibrated at one law was used under another.

The claim, and where it stops #

This essay takes the break search’s two-unit set and point estimate when the break is real. The claims are that with a shift at row 60 of 120 on first-order autoregressive errors the two-unit set contains the true row on 12.0% to 69.3% of draws unwhitened and 22.7% to 75.3% with the window searched, across breaks from 0.3 to 2 standard deviations; that a 95% set would need a threshold of 13.54 to 20.27 units unwhitened and 5.47 to 8.74 whitened; that sets at 21 and 9 units hold the truth on most draws and hold between thirty and sixty-eight of the seventy-three candidate rows, the same width either way; and that searching the window changes the root-mean-square location error by under two rows at every size. Set beside the field’s measured detection rates, a break of one standard deviation is detected on about three draws in ten and placed within five rows on about one in three, and at six tenths of a standard deviation the search places it no better than a uniform guess among the candidate rows.

What stays out, named as a decision: other laws and other break positions — row 60 is the middle of the sample, where a break is easiest to locate; a break in the slope rather than the level; and the bootstrap set, deferred above. The boundary against the charge that depends on the rule is that it measured what the joint search manufactures at a null, and this essay what it finds and how precisely when there is something to find.