Normal Distribution
nothing is called 95% until it has been counted
Field

Two searches over one sample

A break point that has been looked for costs more than a count of parameters says. So does a window chosen from a list, and nothing here had charged for it. A rule that does both is running two searches over one sample, and the charges do not add: the pair manufactures a fifth less than the two apart. What follows is sharper than a correction to a sum — most of what a break search finds under correlated errors is the correlation, so once a whitening has been chosen from the same sample the break's own charge more than halves, and carrying the published one across switches the test off entirely.
Two searches find some of the same luck. What each search reports on a sample with no break in it, and what the two report together, on four dependences. The dashed line is the sum of the two — what a rule charging each search separately would levy — and the two together always come in below it: 19.30, 15.44, 14.24, 26.64 short, on 100%, 99%, 99%, 100% of draws. The shortfall is not a rounding. Under AR(1) at 0.8 it is 19.30 of the 34.70 the break search manufactures on its own, which is more than half of it. Two searches over one sample are looking at the same noise, and the second one has less left to find.

Two searches, one sample

A searched break in a regression manufactures 34.7 of likelihood ratio where a count of coefficients says 11.1. A searched window manufactures 84.0. The two together manufacture 99.4, not 118.7.

 Three charges, and only one of them is a test. What each of three thresholds does to the same decision, under AR(1) at 0.8, against the size of a genuine break in the mean at row 60. A chi-square on the 5 coefficients a split adds — 11.07 — declares a break on 73.6% of samples that have none: it is not a test at all. The break search's own 95% point, 69.6, carried into a rule that also chooses its window, fires on 0.0% of null samples and on 0.0% of samples with the largest break measured — the natural way of combining two published corrections does not lose a little power, it switches the test off. The calibrated charge, 27.2, holds 5.6% at no break and reaches 29.2% at the largest.

The charge that is not a sum

Charging two searches what each costs on its own is conservative, and conservative here means the test never fires. At the largest break measured it declares nothing, on every draw, while a calibrated threshold reaches 29%.

 What a search costs is not a property of that search. The likelihood ratio a searched break in the regression reports, two ways, on every law. On its own — the whole rule being a split of the sample, at no whitening — it averages 34.70 under AR(1) at 0.8, against the 11.07 a chi-square on the five coefficients a split adds would use as a threshold. Inside a rule that also chooses a window from a list of eight, the same search adds only 15.40 — less than half. Most of what a break search finds under correlated errors is the correlation, and a whitening chosen from the same sample has taken it already. A charge measured for one search, carried into a rule that makes two, is not conservative in some harmless direction: it is measuring a different quantity.

A charge that depends on the rule

The break search's charge is 34.7 on its own and 15.4 once a window has been chosen from the same sample. Most of what a break search finds under correlated errors is the correlation, and a whitening has taken it already.

 How often two units contain a break that is there. 300 draws at each of 5 break sizes, a shift from row 60 of 120 on first-order autoregressive errors. The share of draws on which the profile's two-unit set contains the true row: at the fixed, unwhitened window 12.0% at 0.3, 19.0% at 0.6, 31.7% at 1, 51.7% at 1.5, 69.3% at 2; with the window searched 22.7% at 0.3, 30.0% at 0.6, 44.3% at 1, 63.3% at 1.5, 75.3% at 2. A 95% interval would sit at the top of the frame.

A break that is there

Put a real break in the regression at row 60 and the profile's two-unit set still misses it on most draws — it contains the true row on 12% of draws at a small break and 69% at a large one, without whitening; with the window searched as well, 23% to 75%. A set that would contain it on 95% of draws needs a threshold of about 19 units without whitening and about 8.5 with it, and either honest set holds most of the sample: fifty-seven of the seventy-three candidate rows at a break of one standard deviation. Searching the window does not move where the break is found.

 How often each set holds the break. For a shift added to the response from row 60 of 120 on first-order errors at 0.8, 300 samples at each of five sizes and 120 simulations per sample, the coverage of sets for the break row built at a threshold each method supplies. simulated at the truth: 95.0%, 93.7%, 93.7%, 95.0%, 94.0%; estimated break, true persistence: 90.0%, 88.3%, 88.3%, 89.3%, 90.3%; estimated break, persistence matched: 81.0%, 83.3%, 81.3%, 81.3%, 82.3%; estimated break and persistence: 64.0%, 66.0%, 67.7%, 70.7%, 75.7%; a fixed threshold of 21 units: 96.0%, 94.0%, 96.0%, 97.3%, 98.3%; the two-unit set: 12.3%, 19.7%, 35.0%, 49.0%, 66.7%, at shifts of 0.3, 0.6, 1, 1.5, 2. Simulated at the truth the construction is exact; simulated at what one sample can estimate it misses the true row a quarter to a third of the time.

A set calibrated at the estimate

A confidence set for where a regression breaks needs a threshold on the profile likelihood, and no fixed one works: the honest threshold falls as the break grows. Simulating the profile's deficit at the true break, shift and persistence gives a set that covers 93.7% to 95.0% at every break size and narrows from 67 rows to 23. Simulating it at what one sample can estimate gives a set that covers 64% to 76%, and most of the shortfall is one number: a split regression's residuals report a persistence of 0.62 where the errors have 0.8.

All essays