A range that forgets

A range that grows with the lot set a band from the smallest and largest of everything seen so far and let each new unit join it. For any continuous population the units it flags are exactly the new records, the $k$-th flagged with probability $2/(n + k)$ and independently of every other, and that made a promise of at most five flags in the next hundred far cheaper than the fixed range’s: thirty-six starting units instead of a hundred and seven. The price was that it believes every flag. A two-standard-deviation step in the mean is flagged for a few units and then simply absorbed, because the new level’s extremes become the band.

That essay ended on the obvious middle course. A band that remembers only the last $w$ units — a moving window — would flag a step when it arrives, absorb it within about $w$ units, and then flag a return to the old level as a change in its own right. It would forget as well as learn. Whether a window can be chosen to flag a step for long enough to be acted on and then stop, and what a few-misses promise costs it, are both countable, and the first answer is not the one the middle course was hoping for.

A rate that never falls #

The band at unit $k$ is the range of the $w$ units before it. A stable lot’s $k$-th unit falls outside it exactly when it is the largest or smallest of itself and those $w$ — when its rank among $w + 1$ exchangeable values is one of the two extremes. That has probability

$$\frac{2}{w + 1}$$

at every unit, for any continuous population, with nothing estimated. The learning range’s rate falls like one over the count, because its reference keeps growing; the fixed range’s rate is $2/(n + 1)$ at every unit but its flags cluster, because a fixed reference that happens to be narrow is narrow for every unit it judges. The window’s reference is never more than $w$ units old and is renewed one unit at a time.

Its flags also arrive as if independent. Counted over twenty thousand lots of a hundred with $w = 64$, a unit is flagged 3.07% of the time — the exact 2/65 is 3.08% — and a unit that follows a flagged one is flagged 3.06% of the time. The fixed range’s flags cluster by half again, since a second flag after a first has probability $3/(n + 2)$ rather than $2/(n + 1)$: the band that was narrow enough to flag one unit is narrow enough to flag the next. A window forgets its own narrowness as quickly as it forgets anything else.

Both facts rest on exchangeability and on nothing else. Any $w + 1$ consecutive units of a stable lot are equally likely to arrive in any order, which is the whole of what coverage from exchangeability alone needed for a conformal interval and the whole of what the window needs here. A lot whose units are serially correlated is not exchangeable, and there the window’s rate is no longer $2/(w + 1)$: a slowly wandering process sets new local extremes more often than an exchangeable one, and the clustering the tail has found the same mechanism making exceedances arrive in bunches. Every price below is for units that are exchangeable while the process is stable, which is what a stable process means here.

What a few-misses promise costs a window #

Because its count is a function of ranks alone, the window’s promise has the same distribution-free form as the other two. What changes is the price.

To let at most five of the next hundred outside, 95% of the time, the fixed range needs 107 starting units, the learning range 36, and the window 64. At most two: 300, 197 and 236. At most one: 637, 512 and 564. At most ten: 50, 4 and 27. At every allowance the window sits between the two, closer to the learning range when many flags are allowed and closer to the fixed range when few are.

The ordering has a plain reason. The learning range keeps every unit and so grows wider for ever, which is why its later units are cheap; the fixed range keeps only its start; the window keeps a fixed number of the most recent. In a stable lot the window is a fixed range that keeps being re-drawn — the same width on average as a fixed range of $w$, re-sampled continuously. It pays more than the learning range because it never accumulates, and less than the fixed range because a rate of $2/(w+1)$ spread evenly has a lighter tail than the same rate concentrated in lots whose reference happened to be narrow.

Where each band spends its flags #

The promise counts flags over the whole lot. Where they fall is a different property, and it is the one that matters to someone watching the lot run.

The fixed range spends its flags evenly and few of them: 1.81 a lot. The learning range spends 2.62, most of them early, where its reference is thinnest. The window spends 3.06, evenly, and goes on spending them: a flag among the last twenty units of a stable lot happens in 47.3% of lots under the window against 29.3% under the fixed range and 28.2% under the learning range. A band that forgets is a band that keeps finding the lot surprising.

That is the first cost of forgetting, and it is not a defect of the window but a description of it. Late in a long run, a band built on the last sixty-four units knows as much about the process as it did at the start, and no more. Over a thousand units rather than a hundred, the window of sixty-four raises 30.8 flags on average, ten times as many, while the learning range from thirty-six raises 6.7 — its flags thin out as its reference grows, and the window’s never do.

A step, and how long each band keeps flagging it #

The reason to want a window was a step: a change in the process that should be flagged while it is new and then, once it is the process, accepted.

In the ten units after a two-sigma step the fixed range flags 3.13 on average, the learning range 1.93 and the window 1.89. Over all eighty units after it, 25.05, 3.92 and 4.20. In the last twenty units the fixed range is still flagging in 97.5% of lots, the learning range in 24.8% — below its rate with no step at all — and the window in 45.0%, below its 47.3% with no step.

The window absorbs the step as fast as the learning range does, and unit by unit the two are almost indistinguishable. The first unit after the step is flagged in 40.0% of lots by the learning range, 37.9% by the window and 30.8% by the fixed range — the two learning bands start narrower, from thirty-six and sixty-four units against a hundred and seven. By the fifth unit after the step the learning range and the window flag 17.3% and 17.1%; by the tenth, 8.9% and 8.9%; by the twentieth, 5.3% and 5.3%. The fixed range flags 29.8% at the tenth and 32.4% at the twentieth, because the step is still a step to it. The window does not hold the flag for anything like sixty-four units; its flag halves within five, and then the step is inside it. The reason is in how it learns. A unit at the new level that is flagged joins the window exactly as it joins the learning range, and after two or three such units the window’s maximum is a new-level value. The old units the window has not yet forgotten do not hold the band down; they are inside it already. Forgetting the old level is not what absorbs a step. Admitting the new one is, and the window admits it at the same speed as a band that forgets nothing.

At a one-sigma step the picture is the same with smaller numbers. The window flags 0.84 units in the first ten against the learning range’s 0.87 and the fixed range’s 0.78 — a one-sigma step is barely visible to any range of this size at first — and over the eighty units after, 3.12 against 2.66 and 6.24.

The same choice turned up in a different instrument. A quantile over the recent scores gave a conformal interval a window of the last thirty-nine calibration scores, and the window capped what drift could do to coverage because it let the interval follow the process. That is this window’s virtue seen from the other side: a reference that follows the process is a reference that stops objecting to it. Whether that is wanted depends on whether the change is the thing being protected against or the thing being tracked, and no window length can make it both.

A window that refuses to learn from its flags #

If admitting flagged units is what stops the flag, the repair is to admit only the units the band accepts: a window of the last $w$ units that fell inside it. A flagged unit is held out of the reference, as a unit under investigation would be.

That window costs more. Every unit it admits lies inside its current range, so its range can only narrow as old extremes drop out, and in a stable lot its flag rate climbs along the lot — from 3.1% at the first unit to 10.3% at the hundredth when $w = 64$. To keep the promise of at most five flags in a hundred it needs a window of 141, more than the fixed range’s 107.

And under a step it behaves as the fixed range does. In the ten units after a two-sigma step it flags 2.99 against the fixed range’s 3.13, over all eighty 24.46 against 25.05, and in the last twenty it is still flagging in 97.8% of lots. It does what the window was meant to do — it keeps the flag up — by never accepting the new level at all. In a stable lot it is a fixed range whose reference shrinks; under a step it is a fixed range. A range that grows with the lot found that a growing reference admitting only accepted units is the fixed range, exactly; a forgetting reference admitting only accepted units is the fixed range at a higher price.

So there is no window length that flags a step for a while and then stops. A window that admits its flags stops in two or three units at every length; a window that refuses them never stops. The duration of a flag is set by the admission rule, not by the memory.

Where forgetting shows #

Forgetting does show, once, on the way back. When the mean steps up after unit 20 and returns after unit 70, the window of sixty-four has by then replaced most of its old-level units with new-level ones, so the old level looks new to it.

In the ten units after the return the window flags 0.66 units, against 0.18 for the fixed range and 0.17 for the learning range, and against the 0.31 any ten units would draw from it by chance. Over the thirty units after the return, 1.67 against 0.55 and 0.44. The fixed and learning ranges see nothing on the way back, because the old level was always inside them.

The return is seen only if the excursion outlasted most of the window. After an excursion of thirty units — fewer than half of sixty-four — the window flags 0.29 units in the ten after the return, its chance rate, because the old level’s units are still in it. A window is a detector of changes that last longer than itself, and it reports them twice: once, briefly, on the way in, and once on the way out.

What a monitoring specification can take from it #

A window buys an even flag rate, not a lasting flag. It is the right band when the question is whether this part of the run looks like the recent part — when the process is expected to drift slowly and the reference should drift with it — and its price for a few-misses promise is between the other two. It is the wrong band for holding a step in view, which it absorbs as fast as the learning range.

The duration of a flag is a decision about admission. A band that admits its flags stops flagging a step in two or three units whatever its memory; one that admits only accepted units keeps flagging it for as long as it lasts. The second is a fixed range, with or without a window, and a window adds only its own price.

Size the window for the promise over the run that matters. Its rate is constant, so a promise about the next hundred units is a promise about every hundred: all of the next ten found that a band promising every one of a longer future has no ceiling, and a window promising at most five per hundred keeps the same sixty-four units whatever the run length, raising 30.8 flags over a thousand units where a promise of at most five in the whole thousand would need a far wider band.

State the band with its price and its rate along the lot. The same at-most-five promise is kept by three bands whose flags arrive early (learning), evenly and late (window), and evenly and few (fixed). A specification that names only the promise has not said which.

Ninety-three observations and nothing assumed and a range allowed a few misses priced the range as a single fixed object. The same range, moved through a lot, is three different instruments depending on what it keeps, and each is priced exactly by ranks alone. A band allowed a few misses priced the normal-theory band that a window of sample means and spreads would replace, and its misses clustered for the same reason the fixed range’s do.

Exact, and counted #

Exact, for any continuous population: the window’s flag probability at every unit, $2/(w + 1)$; the fixed and learning ranges’ prices for every allowance, 107 and 36 at five.

Counted, on twenty thousand lots of uniform draws: the window’s prices, 64, 236, 564 and 27 at five, two, one and ten flags; its chance of a flag after a flag, 3.06%; the quarantined window’s price of 141. On four thousand normal lots: the flags along a stable lot and after a step, and the return after a fifty-unit excursion, as quoted.

Not claimed: that a window of the last $w$ units is the only way to forget. A reference that weights recent units more heavily without dropping old ones — an exponentially weighted band — forgets smoothly rather than one unit at a time, and its flags are no longer a function of ranks alone. Nor that the step is the only change worth watching for; a slow drift, which the window was built for, is a different question.

Still open: a reference that admits a flag after it is explained #

Between admitting every flag and admitting none is the practice most production lines actually follow: a flagged unit is held out of the reference until it has been investigated, and admitted if it turns out to be sound. That is the quarantine window with a delay, and its behaviour depends on how long investigations take and how often they clear the unit — a step whose first few units are investigated and found sound is admitted a few units late, and then absorbed.

How the duration of a flag depends on the investigation delay, whether a delay of a few units gives a flag long enough to be acted on without the quarantine’s price in a stable lot, and what the promise of at most five flags costs when cleared units rejoin the window, are countable on the same lots and by the same ranks, and have not been counted here.