Normal Distribution
nothing is called 95% until it has been counted
Field

Two searches over different features

A break search and a window search on one sample manufacture less likelihood together than separately, and both read the same residual series. Put four more pairs beside them on a scale whose zero is two searches over independent columns and whose one is a search that contains the other, and the shortfall is a property of the pair: 0.00 for the control, 0.76 for the pair that reads one series twice, exactly 1 for containment — and −0.31 for a break paired with an independent column, where charging the two separately under-charges rather than over-charging.
How much of one search the other has already found. Five pairs of searches on one sample, on a scale whose zero and one are both fixed by construction. Zero is two searches over disjoint sets of independent columns: they remove shares of the residual sum that add, at 0.8 standard errors from exactly additive, and they read 0.004. One is a break search paired with a step column it contains, which reads exactly one on every draw because the step adds nothing at all. Between them: two dictionaries of step columns cut a few rows apart read 0.125, and the pair the earlier field measured — a break and a whitening window, both reading the same residual series — reads 0.762, three quarters of the way to one search containing the other. And below zero, a break paired with a search over independent columns reads -0.306: the joint search finds configurations neither half of it contains, so charging the two separately under-charges.

Two searches that share nothing

Two searches over independent columns remove shares of the residual sum that add exactly. On the scale a chi-square point is quoted on they look super-additive by a fifth of a unit, and none of it is overlap.

 What the second search finds, alone and afterwards. For four of the pairs, what the second search removes on its own and what it removes once the first has already run. The gap between the two is the overlap in absolute terms. Where the searches share nothing the two readings are the same: an independent column removes 0.0261 alone and 0.0260 afterwards. Where one contains the other they are 0.1387 and exactly zero. The pair the earlier field measured sits between: a whitening window removes 0.5033 alone and 0.3120 after a break search has run. This is the earlier field's own reading of its pair, on the share scale rather than in log-likelihood units, and it is the number a rule that runs both searches actually has to charge for.

A search that is already the other

A break search shifts every coefficient after a row, so a step column is one of the directions it can move in. Paired with a dictionary of them it reads exactly one, on every draw, and that fixes the top of the scale.

 The ordering is the same under every law. Each pair's overlap under each of the four laws, over 400 draws apiece. The scale is fixed at both ends by construction: two searches over disjoint sets of independent columns read -0.001, 0.013, -0.002, 0.004, and a break search paired with a step column it already contains reads exactly one under every law. Between them the pair that reads one residual series twice runs 0.763, 0.805, 0.577, 0.752 — lowest under long memory, where a whitening has most to do and the break search has least left to find that the whitening has not taken. The pair that moves most is the pair of step dictionaries, from -0.148 under a moving average to 0.419 under a break; and the pair that is negative is negative under all four.

Three quarters of the way to one search

The pair that started this reads 0.762 on a scale whose one is containment. And the pair that shares nothing but its response reads −0.306, so the sign the earlier field found does not transport at all.

 An overlap that depends on where it is read. Each pair's overlap on the doubled log-likelihood scale — the sum of the two searches' own quantiles less the joint search's quantile, as a share of the smaller — at five quantile levels, over 1200 draws under the first-order autoregression. two dictionaries of independent columns: -0.176 at 0.5, -0.000 at 0.75, 0.238 at 0.9, 0.320 at 0.95, 0.538 at 0.99; a break, and an independent column: -1.211 at 0.5, -0.406 at 0.75, -0.029 at 0.9, 0.205 at 0.95, 0.161 at 0.99; two dictionaries of step columns: -0.397 at 0.5, 0.145 at 0.75, 0.438 at 0.9, 0.541 at 0.95, 0.682 at 0.99; a break, and a step column it contains: 1.000 at 0.5, 1.000 at 0.75, 1.000 at 0.9, 1.000 at 0.95, 1.000 at 0.99; a break, and a whitening window: 0.512 at 0.5, 0.640 at 0.75, 0.716 at 0.9, 0.732 at 0.95, 0.754 at 0.99. Every pair but the containment overlaps more at the 95% point than at the median, the control included.

The overlap at the critical value

Two searches over independent noise columns are additive on average: the joint search finds what the two find separately, to within a fortieth of the smaller. At the point a test actually uses — the joint statistic's 95% quantile — the same pair overlaps by 0.320, and a rule that adds the two searches' own critical values rejects 1.4% of true nulls instead of 5%. Nothing about the searches causes it. Quantiles of a sum do not add, and on every pair in the field the overlap at the critical value is larger than the overlap the field measured.

 Four rules for a break and a column. How often each of four rules says something is there, for the pair of searches over a break and a column, under four truths, 400 draws each under a first-order autoregression at 0.8 with every threshold simulated on 1200 null draws. nothing there: joint, calibrated 6.3%, summed thresholds 6.3%, Bonferroni 6.3%, either, uncorrected 12.0%. a shift from row 60: joint, calibrated 65.3%, summed thresholds 64.0%, Bonferroni 55.0%, either, uncorrected 69.3%. a column enters: joint, calibrated 11.8%, summed thresholds 10.8%, Bonferroni 73.0%, either, uncorrected 82.5%. both: joint, calibrated 70.5%, summed thresholds 69.0%, Bonferroni 75.8%, either, uncorrected 84.8%. The first block is each rule's size.

The power a calibration buys

Charging two searches separately makes a joint test conservative, and calibrating the joint statistic on its own null repairs that. Against a real break, on the pair of a break search and a window search, the repair is worth 47 points of power — 62.0% against 14.5%. On the pair of a break search and a column search it is worth nothing, and worse: a real column is found by the calibrated joint test on 11.8% of samples and by a plain Bonferroni split on 73.0%, because a joint statistic takes its threshold from the noisier of its two searches.

All essays