A tail the law gives the pair

Two searches run on one sample over-charge, if they are charged separately, by the excess: the sum of what each finds alone less what the joint search finds. Two effects in one number split that excess into two parts that move independently. The overlap is ground both searches find — the sum of the two alone less a sequential run, in which the first search is pinned at its own answer and the second is searched. The interaction is what only the joint search reaches — the joint search less the sequential run, never negative. A split that depends on the order found that which search is pinned first changes the split, so every pair has two of them.

The split at the critical value read the same identity at quantiles of the three statistics rather than at their means, because a test uses the 95% point and not the average. Its finding was that the two parts behave differently in the tail. The overlap climbs — from nothing on average to 2.44 at the 95% point for two independent dictionaries, purely from the arithmetic of adding two tails — while the interaction stays within a unit or two of its average on every pair. A rule that simulates the joint statistic, it concluded, should expect its over-charge to be mostly the arithmetic of tails.

That conclusion rested on one law, a first-order autoregression at 0.8, and the essay said what would change it. The arithmetic of adding tails does not depend on the law. The interaction does, because it is a property of what the joint search can reach, and the earlier field had found the break-and-window pair’s joint behaviour moving by half between laws. If the interaction grew into the tail under some law, that pair would have a tail of its own that no reading of averages could predict.

The same split under three laws #

The construction is the field’s: a regression on four persistent predictors at 120 rows, five pairs of searches — two independent dictionaries of noise columns, a break with a column, two interleaved step dictionaries, a break with step columns, a break with a whitening window — and the three suprema per pair, each search alone, the joint search, and the sequential run in each order. Twelve hundred draws under each of three laws for the errors: the field’s first-order autoregression, a five-period moving average with nothing past the fourth lag, and long memory at a fractional difference of 4/9. Everything is read at the 50%, 75%, 90%, 95% and 99% points.

In the order the field reads — the break searched first and pinned, then the window — the break-and-window pair’s interaction stays low under every law. Under the autoregression it runs 1.57 at the median, 3.22 at the 95% point and 2.75 at the 99th, around an average of 1.46; under the moving average 2.00, 2.63 and 0.60 around 1.88; under long memory 0.53, 0.46 and 1.35 around 0.58. The earlier essay’s finding transports to both new laws in this order.

The other order is where the law shows. With the window searched first and pinned, then the break, the interaction under the moving average is almost nothing everywhere — 0.16 at the median, 0.29 at the 95% point, 0.00 at the 99th. Under the autoregression it rises modestly into the tail, from 0.98 to 2.55. Under long memory it climbs: 2.01 at the median, 4.75 at the 90% point, 6.07 at the 95% and 7.85 at the 99%, against an average of 1.99. At the critical value a test would read, the configurations only the joint search reaches are worth three times what they are worth on an average draw, and at the 99th percentile four times.

So the lead’s second possibility is the one that happened, in one order. Under long memory the break and the window have a tail of their own.

Why one order and one law #

One reading accounts for it, and it is offered as a reading rather than measured. The interaction in a given order is what the joint search finds beyond the sequential run that searches the second quantity after pinning the first at its own answer. With the window pinned first, the window is chosen to whiten the errors as though there were no break; under long memory, whose autocorrelation is still 0.57 at the twentieth lag, the widest bands win that search on most draws. The break is then searched through that wide whitening. The joint search is free to choose a narrower band together with a break — and on the draws where a break-shaped wander in the errors is large, a narrower band that leaves that wander in the residuals, with a break placed to absorb it, beats the wide band by a great deal. Those are exactly the draws in the joint statistic’s upper tail.

With the break pinned first the window search runs after the break has already absorbed the largest wander, and there is little left for a joint choice to find. Under the moving average there is no long wander to absorb, the window search’s answer barely depends on whether a break is present, and both orders’ interactions are small. The tail belongs to the combination of a dependence that looks like breaks and a search order that whitens before it looks.

That reading is consistent with the two orders’ averages as well: under long memory the window-first interaction averages 1.99 and the break-first one 0.58, so the order matters on average too, and the tail sharpens a difference that was already there.

Every other pair stays put #

In the field’s order, no pair’s interaction moves by as much as three units between its average and the critical value under any law. The two independent dictionaries’ interaction is under a fifth of a unit at both. The break with a column moves from 0.34 to 0.00 under the autoregression, 0.39 to 0.16 under the moving average and 0.35 to 0.01 under long memory. The step dictionaries, the pair with the largest interaction of all, move from 4.59 to 4.75, 4.50 to 3.88 and 2.88 to 2.07. The break-and-window pair moves furthest, from 1.46 to 3.22 under the autoregression, and that is the field’s original measurement.

So for the order a practitioner would naturally use — search the thing of interest, then tune the nuisance — the earlier rule survives three laws: simulate the joint statistic and expect the over-charge in the tail to be the overlap.

The overlap is where the law moves things #

What does move with the law is the part that was supposed to be arithmetic. The independent dictionaries’ overlap at the 95% point is 2.44 under the autoregression, 3.26 under the moving average and 2.46 under long memory: the arithmetic of adding two tails, moved a little by the law through the shape of each search’s own null. The pairs that share ground move much more. The step dictionaries’ overlap at the 95% point is 31.02 under the autoregression, 20.25 under the moving average and 41.79 under long memory — a factor of two between laws. The break-and-window pair’s is 53.79, 40.46 and 41.48. The break with step columns, whose interaction is identically zero because a step column is a break, has an overlap equal to the step search’s own 95% point under every law: 33.35 under the moving average, 50.88 under the autoregression.

That is a caution about reading the earlier essay’s conclusion as a rule with a number in it. “Expect the over-charge in the tail to be the overlap” holds under every law here, in the field’s order. “Expect the overlap to be about thirty units” does not hold under any two of them. A threshold calibrated by simulation under one law and transported to another carries the wrong overlap, which is the transport problem the charge that is not a sum met on the same pair.

The pair with the largest interaction #

The two interleaved step dictionaries are the pair built to have a large interaction: each column has a near-twin a couple of rows away in the other dictionary, so the joint search can combine two steps into a shape neither search alone can reach. Their interaction is the largest on the field’s scale, and it is nearly level from the median to the 95% point under every law — 4.87 to 4.75 under the autoregression, 4.62 to 3.88 under the moving average, 2.97 to 2.07 under long memory — before falling at the 99th point, where both statistics are dominated by the overlap. The law sets its height, highest under the autoregression and lowest under long memory, and does not give it a tail.

That separates the break-and-window result from a general statement that long memory inflates interactions. Long memory lowers the step pair’s interaction at every level. What it does to the break and the window is specific to a pair whose second search, run first, can mistake a long wander for a dependence to be whitened away.

How often the joint search finds nothing new #

The interaction at a quantile is a difference between two quantiles, and a reader can mistake it for the quantile of a difference. The per-draw interaction — the joint search’s gain less the sequential run’s, draw by draw — is a different object, and it says something the quantiles cannot: how often the joint search reaches nothing the sequential run did not.

In the field’s order, the two independent dictionaries’ joint search finds exactly what the sequential run finds on 85% to 88% of draws under all three laws, and the break with a column on 68% to 69%. Those are pairs whose searches share almost nothing, and the joint search usually lands where the sequential one did. The step dictionaries, built to interact, find something new on more than half of draws under every law — they agree on only 39% to 47%.

The break and the window are where the law shows again. Under the autoregression the joint search finds nothing new on 26.7% of draws, under the moving average on 17.8%, and under long memory on 53.9%. In the field’s order, long memory makes the joint search agree with the sequential one twice as often as the autoregression does. On the reading offered above, the break, searched first, absorbs the wander, and the window chosen after it is the window the joint search would have chosen. That is the same mechanism as the window-first tail, seen from the other side: under long memory what matters is which search gets to the wander first.

The per-draw interaction’s own 95th percentile is a third reading again, and it is much larger than the interaction at the 95% point: 5.62 under the autoregression, 6.06 under the moving average and 2.91 under long memory for the break and the window in the field’s order, against quantile differences of 3.22, 2.63 and 0.46. The draws on which the joint search finds the most are not the draws on which the joint statistic is largest. A test reads the second; the first is what a reader inspecting single samples would see, and it is from nearly twice to six times bigger. Neither reading is wrong, but they answer different questions, and only the quantile difference belongs in a critical value.

What this changes for a practitioner #

The rule the earlier essay offered — simulate the joint statistic, and expect its tail over-charge to be overlap — stands in the order that searches the quantity of interest first, under every law measured. What changes is that the order is part of the rule. A practitioner who whitens first and then searches for a break, under a dependence with long memory, is running the order whose interaction has a tail, and a critical value simulated for the joint search is then the only one that charges for it; a critical value simulated for the sequential procedure would miss the configurations only the joint search reaches, at exactly the quantiles a test reads.

How the numbers were made #

Twelve hundred draws under each law, each with a fresh set of dictionaries drawn from a stream of its own, and on each draw every pair’s suprema computed exactly as the field defines them: the break over the middle sixty per cent of rows with the regression’s five coefficients changing, the dictionaries over six columns each, the window over eight widths with its determinant and a unit charge a lag. The sequential runs pin the first search at the answer it gives alone and search the second. Quantiles are of the twelve hundred values of each statistic, and the split at a quantile is exact by construction: overlap less interaction equals the excess at every level, to machine precision, and the interaction is never negative because the joint search is never below the sequential one.

Checked on the measurement: under long memory the window-first interaction at the 99% point is more than three times its own average and more than three units above its median; and in the field’s order every live pair’s interaction at the 95% point is within three units of its average under every law.

Still open: whether the order a practitioner uses is the safe one #

The finding is that the order matters in the tail under long memory, and the order with the tail is the one that tunes the nuisance first — which is how a regression on dependent errors is usually run in practice: whiten, then search. The measurement that would settle whether that habit is costly is the one the power a calibration buys made for the joint test: the size and power of a break test read through a window chosen first, under long memory, calibrated on the joint statistic and on the sequential one. If the sequential statistic’s tail is short of the joint one’s by the six units the window-first interaction carries at the 95% point, a test calibrated on the sequential run would over-reject by a measurable amount. That has not been run.

What is claimed here, and what is not #

This essay takes the field’s split of an excess into overlap and interaction, at quantiles, under three laws. The claims are that in the field’s order every live pair’s interaction at the 95% point is within three units of its average under the autoregression, the moving average and long memory; that with the window searched first, the break-and-window pair’s interaction under long memory runs from 2.01 at the median to 6.07 at the 95% point and 7.85 at the 99th against an average of 1.99, while under the moving average it is under 0.3 at every level and under the autoregression rises from 0.98 to 2.55; that the step dictionaries’ interaction is nearly level into the tail under every law at a height the law sets; and that the overlap at the 95% point moves with the law by up to a factor of two for the pairs that share ground.

What stays out, named as a decision: other sample sizes, at which long memory’s wander is a different share of the series; the test built on either order, deferred above; and other pairs, since every interaction here belongs to one of the field’s five.

The boundary against the split at the critical value is that it read the split at quantiles under one law and this essay under three, and finds its conclusion holding in its own order and failing in the other. Against a split that depends on the order, which found the split moving with the order on average, this essay finds the order deciding whether the interaction has a tail.