Two effects in one number

The field that built a scale for how much two searches share reports one number per pair. Call the share of the residual sum each search removes on its own δA and δB, and the share the joint search removes δC; the reading is the excess, δA + δB − δC, which is what a rule charging the two searches separately over-charges.

The ladder it builds runs from 0.004 for two searches over independent columns, through 0.125 and 0.762, to exactly 1 for a search that contains another — and to −0.306 for a break paired with an independent column, where charging separately under-charges.

Its own last paragraph says what that number cannot do. It is the net of two effects, and a pair reading zero may have neither of them or both of them in equal measure.

The two effects #

Overlap. Two searches can be looking for the same feature. A break search and a search over step columns are both asking when the series changes, so once the break search has run there is less for the step search to find, and the joint search finds less than the sum. This makes the excess positive.

Interaction. A joint search over two things at once can reach configurations neither of its slices contains. The best break position when a noise column is in the design is not the best break position without it, so a search that moves both at once finds something a search that moves one and then the other does not. This makes the excess negative.

Both are real, both are present in every pair, and one number cannot say which is which.

Why one number could not do it #

Before the construction, it is worth being clear that this is not a case of a measurement being too noisy.

The excess is measured well. On the control it is 0.000011 with a standard error of 0.000286 over three hundred draws, so it is inside a twentieth of a standard error of exactly zero — as clean a null reading as this collection produces. On the break-and-window pair it is 0.191122 ± 0.005349, thirty-six standard errors from zero.

The problem is not precision, it is identification. δA + δB − δC is one number computed from three suprema, and the two effects it is a net of both live inside those same three suprema. No amount of averaging separates them, because they are not two noisy versions of one thing; they are two things that were added before anybody looked.

That is why the repair is an extra measurement rather than a longer sweep. The fourth supremum — the sequential one — is a quantity the three do not contain, and it is the only one that splits them.

The construction #

Run the first search alone and note the answer it gives. Then run the second search with the first held at that answer.

That is one extra supremum per pair per draw, and it is the missing term:

$$\text{overlap} = \delta_A + \delta_B - \delta(\text{A pinned}) \qquad \text{interaction} = \delta_C - \delta(\text{A pinned})$$

$$\text{excess} = \text{overlap} - \text{interaction}$$

The third line is an identity rather than a model. The pinned supremum appears in both components with opposite signs and cancels, so the split adds back to the original number exactly, on every draw, at no tolerance at all.

That is what makes it worth making. A decomposition that only nearly adds up is not a decomposition; it is two measurements with a coincidence between them.

The picture is the identity rather than an illustration of it. Every bar in it is drawn as two segments meeting at the value the pair actually measured, so a gap or an overlap between them would be visible immediately and there is never one. Dragging the number of draws moves both segments — the estimates are noisy, and at three hundred draws they are noticeably noisy — without ever moving the point where they meet, because that point is the measured excess and the split is computed from it rather than alongside it.

What “pinned” means, exactly #

The word is doing precise work and the three searches are pinned in three different ways, so it is worth writing out.

A break is pinned at a row. The joint search normally sweeps every admissible break position; pinned, it evaluates one.

A window is pinned at a width. The joint search normally whitens at every width on the list; pinned, it whitens at one.

A dictionary column is pinned at an index. The joint search normally tries every column and the option of taking none; pinned, it takes the named column.

Each of those restricts the joint search’s own loop rather than running a different procedure, which is what makes the inequality below hold by construction rather than by argument. The pinned run is the joint run with a smaller feasible set, and the smaller set contains the point the first search chose.

The third case has a consequence worth flagging now: pinning a column removes the option of taking none, and that turns out to make one of the five pairs’ second orders impossible. The third essay is where that lands.

The inequality it rests on #

Everything above depends on one claim: the joint search’s supremum is at least the sequential one’s.

It has to be. The pinned point — the first search at its own answer, the second at whatever it likes — is a point in the joint search’s own feasible set, since both searches range over the same grids in both runs. So the joint supremum cannot be below the pinned one.

That is a statement about the code rather than about the world, and it is asserted as one: on every draw and in both orders, the raw log-likelihood slack between the joint supremum and the pinned one is checked to be non-negative at exactly zero tolerance. Over three hundred draws and five pairs the worst slack is 0.

An interaction that came out negative would mean the two searches are not searching the same space in the two runs — that a grid changed, or an index slipped — and there is no other way for it to happen. It is the one place this construction could be silently wrong and produce a decomposition that adds up perfectly and means nothing.

The five pairs, split #

Over three hundred draws under a first-order autoregression:

Three of the five rows say something the excess column alone does not.

The control’s zero is two effects cancelling. Its overlap and its interaction are each several times the number they subtract to, which is the next essay’s subject.

The containment rung is all overlap. A break paired with a dictionary of step columns has an interaction of exactly zero — not small, zero, on every draw — because the joint search never moves the break off its own answer. That is the rung the earlier field fixes by construction, and it arrives here as an identity rather than as a measurement.

And the negative rung is interaction beating overlap. A break paired with an independent column has an interaction of 0.002364 against an overlap of −0.004395, which is what puts its excess below zero.

The one that reads negative on both #

The break-and-column pair is worth a paragraph, because its overlap is negative and the word overlap does not obviously admit a negative value.

It can. The overlap is what the second search loses by having the first already run at its own answer, and there is no reason that has to be a loss. A break search that has fitted a break has changed the residual series the column search reads: the two segments are separately centred, so a column that was mildly useful before the break was fitted may be more useful after it.

When that happens the second search finds more after the first has run than it found alone, the overlap comes out negative, and the pair is complementary in the sequential sense as well as in the joint one.

So the sign of the overlap is a finding rather than an accounting convention, and it is the first thing the excess column hides: a pair that under-charges could be doing so because the joint search reaches further than either slice (interaction) or because running one search makes the other more productive (negative overlap). This pair does both.

What the components mean, one at a time #

It is worth reading the two quantities in words once, because “overlap” and “interaction” are both used loosely elsewhere and here they name two specific suprema.

The overlap is a loss to the second search. δA + δB is what the two searches find if each is run on the untouched sample. δ(A pinned, B searched) is what they find if the first is run, its answer is kept, and the second is run against it. The difference is what the second search stopped being able to find once the first had taken what it took. A pair with a large overlap is a pair whose searches are looking at the same feature.

The interaction is a gain to the joint search. δC is what a search that moves both at once finds. δ(A pinned, B searched) is what a search that moves the first, fixes it, and then moves the second finds. The difference is what the joint search buys by being allowed to move the first search off its own best answer. A pair with a large interaction is a pair whose two searches are entangled: the best value of one depends on the value of the other.

Both are shares of the residual sum, both are on the same scale as the excess they subtract to, and neither is bounded in sign except that the second cannot be negative. Neither is bounded above by anything either: the containment rung’s overlap is 0.136647 on searches that remove 0.24975 and 0.13665 on their own, which is the whole of the smaller search’s finding and is what containment means on this scale.

How often the joint search actually moves is reported beside them: the share of draws on which the joint supremum is not the pinned one. It is 0.13 for the control, 0.29 for the break-and-column pair, 0.58 for the two step dictionaries, 0.78 for the break-and-window pair, and exactly nothing for the containment rung.

Why the shares and not the log-likelihoods #

One inherited choice is worth restating because it makes the difference between a decomposition and an artefact.

A gain is a difference of log-likelihoods, so it is a log of a ratio of residual sums, and logs of ratios do not add when the reductions do: two searches that each remove a tenth of the residual sum remove a fifth of it together, and the second one’s log gain is then larger than it was alone. A rule that adds two charges on the log scale is wrong by that amount before any overlap.

The earlier field establishes it and repairs it: everything is read on the share of the residual sum removed, $\delta = 1 - e^{-G/n}$, on which two searches whose directions are orthogonal are exactly additive, and two independent columns are orthogonal in expectation by construction.

Both components here inherit that scale. On the log scale the control’s excess is not zero and neither of its components would be interpretable, so the whole split would be measuring the convexity of a logarithm.

The break-and-window pair, which is where the line started #

The bottom row of the table is the pair an earlier field measured on its own, and it is worth reading with the split attached because it is the one anybody would meet in practice: a series is searched for a break, and the same series is searched for how wide a whitening window its errors need.

Its excess is 0.191122, its overlap 0.197114 and its interaction 0.005991. So it is very nearly all overlap: the two searches are looking at the same thing, and the joint search’s freedom to move the break off its own answer buys about three per cent of what the sharing costs.

That is a comfortable result and it should not be over-read. The joint search moves the break on 78% of draws — the highest share of any pair here — so it is not that the two searches are independent enough to leave each other alone. It is that when the joint search moves the break, it moves it somewhere that is barely better.

The reading a practitioner wants from that is specific. If two searches are nearly all overlap, then running them sequentially — break first, then window — costs almost nothing against running them jointly, and a charge derived from the sequential run is nearly the right charge. If they were largely interaction, sequential and joint would be different procedures and a charge for one would not do for the other.

The excess cannot say which case a pair is in, and the split can.

Ranked by character rather than by size #

The table’s rows are in the order the ladder puts them — by excess, from a control at zero to a pair at 0.191 — and dividing the two components gives a second ordering that is almost the reverse.

The interaction as a share of the overlap is 0.98 for the control, −0.54 for the break with an independent column, 0.68 for the two step dictionaries, 0.00 for the containment rung and 0.03 for the break with a window. That ratio is scale-free: it says what kind of pair this is rather than how much sharing there is, and it is available on any pair whose two searches can be pinned.

Read that way the ladder comes apart. The control, whose excess is the smallest number in the table, is the most interaction-dominated pair on it; the containment rung, near the top of the excess ladder, is the least, at exactly zero by construction. The pair with the largest excess is the third least entangled. So the quantity the earlier field ranks its pairs by and the quantity that says whether the two searches are entangled are near enough independent, and neither can be inferred from the other.

That gives the split a use beyond diagnosing the control. A charge derived from a sequential run is right for a pair whose ratio is near zero and wrong for one whose ratio is near one, whatever the excess is. The break-and-window pair at 0.03 can be charged sequentially and lose three per cent of the sharing; the two step dictionaries at 0.68 cannot, and their excess of 0.014 — a tenth of the break-and-window pair’s — gives no warning of it.

How often it moves and how much it gains are different questions #

The share of draws on which the joint search does not land on the pinned point — 0.13, 0.29, 0.58, 0.78 and exactly nothing — looks like a second measurement of the interaction, and dividing one by the other says it is not.

The interaction per move runs 0.0039, 0.0081, 0.0517, 0.0077 and undefined: a spread of thirteen-fold across four pairs, so the move rate and the gain are not proportional and neither substitutes for the other. Ranked by move rate the pairs run window, step dictionaries, break and column, control; ranked by interaction they run step dictionaries, window, break and column, control. One adjacent transposition separates the two orderings, and it is the top pair.

That transposition is the essay’s own observation about the break-and-window pair generalised: it moves the break most often of any pair and gains least per move. The two step dictionaries move less often and gain five times as much in total, because when a search over one dictionary shifts a search over the other, it shifts it somewhere genuinely better.

So a diagnostic built on the move rate alone — does the joint search ever disagree with the sequential one — would have flagged the break-and-window pair hardest and the pair that actually needs a joint charge second. The frequency of disagreement is cheap to compute and is not the quantity; the size of what the disagreement buys is, and it takes the sixth supremum to get it.

How much this costs #

Six suprema per pair per draw rather than four: the base fit with nothing switched on, each search alone, both together, and the two sequential runs. Half again on the most expensive part of the earlier field’s loop.

That is worth stating because it is the whole reason the split was not made when the ladder was built. The suprema are not free — the break sweep is a pass over every admissible row for every combination of the other searches’ choices, and the window sweep is a whitening per width — so a fifty per cent increase is a real decision rather than a rounding error.

What it buys is that every rung of the ladder now says which of two things it is made of, and one of them turns out to be a rung whose number was the residue of two effects an order of magnitude larger.