Normal Distribution
nothing is called 95% until it has been counted
Concept

Supremum statistic — where it appears

A statistic formed by maximising over a nuisance parameter rather than by fixing one. Its distribution is the distribution of a maximum, so it is non-negative by construction and sits well above what a count of restrictions would predict, on both its mean and its upper tail.

Named by 16 essays across 6 fields — each of them below, with the objects they name alongside it.

The profile a break point is chosen from. One sample of 120 rows under a break in the persistence, fitted as two first-order regimes at every admissible break point. The maximum is at row 78, where the true break is at 60. The shaded band is every break point within two log-likelihood units of the best one — 8 of the 73 positions searched, which is 11% of the range. The horizontal line is the one-regime fit the search is compared against; the whole profile is above it, at every position, which is the point: a maximum over 73 candidates is above the null by construction and not by evidence.

A break that was looked for

A two-regime whitening finds its change point by maximising a profile, and then reads a criterion that counts parameters. Under no break there is no parameter to count, because every position describes the same model.

charged · Break point What each pair is made of. Each pair of searches, over 300 draws, split into the two effects its excess is the difference of. The overlap is what the second search loses by having the first already run at its own answer; the interaction is what the joint search finds by moving the first off it. They subtract to the excess exactly, on every draw, because the pinned supremum cancels. Two disjoint dictionaries of independent columns read an excess of 0.000011 and are made of 0.000514 and 0.000503. A break paired with a dictionary of step columns has an interaction of exactly 0 and is all overlap. And a break paired with an independent column has an overlap of -0.004395 against an interaction of 0.002364, which is what puts its excess below zero.

Two effects in one number

How much two searches over one sample share is measured as the net of two things — ground both of them find, and configurations only the joint search reaches. One extra supremum per draw separates them exactly.

separate · Break point Two searches find some of the same luck. What each search reports on a sample with no break in it, and what the two report together, on four dependences. The dashed line is the sum of the two — what a rule charging each search separately would levy — and the two together always come in below it: 19.30, 15.44, 14.24, 26.64 short, on 100%, 99%, 99%, 100% of draws. The shortfall is not a rounding. Under AR(1) at 0.8 it is 19.30 of the 34.70 the break search manufactures on its own, which is more than half of it. Two searches over one sample are looking at the same noise, and the second one has less left to find.

Two searches, one sample

A searched break in a regression manufactures 34.7 of likelihood ratio where a count of coefficients says 11.1. A searched window manufactures 84.0. The two together manufacture 99.4, not 118.7.

twice · Break point How much of one search the other has already found. Five pairs of searches on one sample, on a scale whose zero and one are both fixed by construction. Zero is two searches over disjoint sets of independent columns: they remove shares of the residual sum that add, at 0.8 standard errors from exactly additive, and they read 0.004. One is a break search paired with a step column it contains, which reads exactly one on every draw because the step adds nothing at all. Between them: two dictionaries of step columns cut a few rows apart read 0.125, and the pair the earlier field measured — a break and a whitening window, both reading the same residual series — reads 0.762, three quarters of the way to one search containing the other. And below zero, a break paired with a search over independent columns reads -0.306: the joint search finds configurations neither half of it contains, so charging the two separately under-charges.

Two searches that share nothing

Two searches over independent columns remove shares of the residual sum that add exactly. On the scale a chi-square point is quoted on they look super-additive by a fifth of a unit, and none of it is overlap.

apart · Criterion What the second search finds, alone and afterwards. For four of the pairs, what the second search removes on its own and what it removes once the first has already run. The gap between the two is the overlap in absolute terms. Where the searches share nothing the two readings are the same: an independent column removes 0.0261 alone and 0.0260 afterwards. Where one contains the other they are 0.1387 and exactly zero. The pair the earlier field measured sits between: a whitening window removes 0.5033 alone and 0.3120 after a break search has run. This is the earlier field's own reading of its pair, on the share scale rather than in log-likelihood units, and it is the number a rule that runs both searches actually has to charge for.

A search that is already the other

A break search shifts every coefficient after a row, so a step column is one of the directions it can move in. Paired with a dictionary of them it reads exactly one, on every draw, and that fixes the top of the scale.

apart · Criterion Three charges, and only one of them is a test. What each of three thresholds does to the same decision, under AR(1) at 0.8, against the size of a genuine break in the mean at row 60. A chi-square on the 5 coefficients a split adds — 11.07 — declares a break on 73.6% of samples that have none: it is not a test at all. The break search's own 95% point, 69.6, carried into a rule that also chooses its window, fires on 0.0% of null samples and on 0.0% of samples with the largest break measured — the natural way of combining two published corrections does not lose a little power, it switches the test off. The calibrated charge, 27.2, holds 5.6% at no break and reaches 29.2% at the largest.

The charge that is not a sum

Charging two searches what each costs on its own is conservative, and conservative here means the test never fires. At the largest break measured it declares nothing, on every draw, while a calibrated threshold reaches 29%.

twice · Break point What a search manufactures, law by law. The average likelihood ratio a search over 120 rows reports, on each of the four laws, over 400 draws. Three of them have no break at all and report 5.080, 4.839 and 4.748; the fourth has one and reports 8.757, so the real break is worth only 3.677 beyond what the search would have found anyway. The dashed line is 2, which is what an information criterion charges for one extra parameter. A search costs about two and a half of them, and the number is a measurement rather than a count.

What a search costs in parameters

An information criterion's penalty is an estimate of the optimism a fit carries. For a break point the optimism can be measured and cannot be counted, and it comes to about two and a half parameters.

charged · Break point Both halves grow; the difference does not. The control pair's two components and their difference, against how much each of its two searches can find, over 1200 draws at each dictionary size. Two disjoint sets of independent columns are additive at every size — the excess stays inside a standard error or two of zero throughout — and it is not because there is nothing there. The overlap grows from 0.000112 at two columns to 0.000870 at ten, a factor of 7.76, and the interaction grows with it, staying within a factor of two of the overlap at every size. Two searches competing for one residual sum share ground and find configurations neither has alone, in almost equal measure, and their difference is what the earlier field's scale calls zero.

What a zero is made of

Two disjoint dictionaries of independent columns read an excess of 0.000116 and are made of an overlap of 0.000583 and an interaction of 0.000467. The control the whole scale is anchored on reads zero because two effects cancel.

separate · Break point What a search costs is not a property of that search. The likelihood ratio a searched break in the regression reports, two ways, on every law. On its own — the whole rule being a split of the sample, at no whitening — it averages 34.70 under AR(1) at 0.8, against the 11.07 a chi-square on the five coefficients a split adds would use as a threshold. Inside a rule that also chooses a window from a list of eight, the same search adds only 15.40 — less than half. Most of what a break search finds under correlated errors is the correlation, and a whitening chosen from the same sample has taken it already. A charge measured for one search, carried into a rule that makes two, is not conservative in some harmless direction: it is measuring a different quantity.

A charge that depends on the rule

The break search's charge is 34.7 on its own and 15.4 once a window has been chosen from the same sample. Most of what a break search finds under correlated errors is the correlation, and a whitening has taken it already.

twice · Break point The split depends on the order. How much two searches share, measured both ways round, over 300 draws. Pinning the first search at its own answer and searching the second gives one overlap; pinning the second and searching the first gives another. A break paired with an independent column reads -0.004395 one way and 0.002163 the other, at 8.70 paired standard errors and on opposite sides of zero. The excess the two components subtract to is the same in both orders by construction, so what changes is only how it is attributed. There is no order-free way to say which of two searches found ground both can reach, and the two orders bracket it.

A split that depends on the order

Run the second search first and pin that instead, and the same draw gives a different overlap and a different interaction — with the same difference. And one pair has no second order at all.

separate · Break point The ordering is the same under every law. Each pair's overlap under each of the four laws, over 400 draws apiece. The scale is fixed at both ends by construction: two searches over disjoint sets of independent columns read -0.001, 0.013, -0.002, 0.004, and a break search paired with a step column it already contains reads exactly one under every law. Between them the pair that reads one residual series twice runs 0.763, 0.805, 0.577, 0.752 — lowest under long memory, where a whitening has most to do and the break search has least left to find that the whitening has not taken. The pair that moves most is the pair of step dictionaries, from -0.148 under a moving average to 0.419 under a break; and the pair that is negative is negative under all four.

Three quarters of the way to one search

The pair that started this reads 0.762 on a scale whose one is containment. And the pair that shares nothing but its response reads −0.306, so the sign the earlier field found does not transport at all.

apart · Criterion What a second break adds. Over 200 draws, the likelihood ratio a search over one break point reports, and how much more a search over an ordered pair adds on top of it. Under AR(1) at 0.8, which has no break at all, the first search manufactures 5.697 and the second adds 4.278. Under a law with exactly one break — where a second one is as absent as the first was in the row above — the first search reports 9.442 and the second still adds 5.800. Searching for something that is not there costs the same whether or not something else was there to find.

A second break on a flat profile

Searching a hundred and twenty rows for one change point manufactures five units of likelihood. Searching for a second manufactures four more, on a series that has at most one — and on a profile whose whole range is under seven.

charged · Break point The overlap climbs and the interaction does not. The over-charge at each quantile level split into overlap — the two searches' own quantiles less the sequential run's — and interaction — the joint search's quantile less the sequential run's — for three pairs over 1200 draws. two independent dictionaries: overlap -0.40, 0.01, 1.55, 2.44, 5.72; interaction 0.07, 0.01, 0.11, 0.06, 0.00. two step dictionaries: overlap -0.73, 8.19, 22.53, 31.02, 47.75; interaction 4.87, 4.55, 4.90, 4.75, 1.10. a break and a window: overlap 17.88, 29.52, 43.01, 53.79, 71.61; interaction 1.57, 1.29, 1.19, 3.22, 2.75, at the 50%, 75%, 90%, 95% and 99% points.

The split at the critical value

Split the over-charge of two searches at the joint statistic's 95% point rather than at its mean, and the two components behave differently. The interaction — what only the joint search can reach — stays within two units of its average at every level up to the 95% point. The overlap climbs from nothing to most of the over-charge, on two searches over independent columns as much as on searches that share ground. At the critical value the split's overlap has stopped meaning shared ground: it is mostly the arithmetic of adding tails, and the identity that defines it hides that.

separate · Break point One order's interaction climbs under long memory. The break-and-window pair's interaction — the joint search's quantile less the sequential run's — at five quantile levels, in the field's order (break first, solid) and the other (window first, dashed), under three laws, 1200 draws each. first-order autoregression: break first 1.57, 1.29, 1.19, 3.22, 2.75; window first 0.98, 1.26, 1.53, 2.55, 2.14. five-period moving average: break first 2.00, 1.88, 2.29, 2.63, 0.60; window first 0.16, 0.21, 0.26, 0.29, 0.00. long memory: break first 0.53, 0.80, 1.05, 0.46, 1.35; window first 2.01, 1.62, 4.75, 6.07, 7.85, at the 50%, 75%, 90%, 95% and 99% points. Under two of the laws, and in the field's order under all three, the interaction stays within a few units of its median. Under long memory with the window searched first it climbs from 2.01 to 7.85, against an average of 1.99; the same order's averages under the other two laws are 0.94 and 0.17.

A tail the law gives the pair

Split the over-charge of two searches at their critical value and the interaction — the configurations only the joint search reaches — stays near its average while the overlap climbs. That held under one law. Under a five-period moving average and under long memory it holds for every pair in the order the field reads, to within three units. In the other order, one pair acquires a tail of its own: with the window searched first and the break second, the interaction under long memory climbs from 2.0 at the median to 7.9 at the 99th percentile, four times its average.

separate · Break point Three detectors for one departure, all at 5%. How often each of three checks on the calibration scores fires, against the size of the drift, with every critical value simulated under no drift so that all three sit at 5.0% exactly. The incumbent — a rank comparison of the first half of the scores against the second — reaches four-in-five power at a growth factor of 4.31. Reading each score's rank against its position reaches it at 2.65, and the largest running departure of the scores from their mean at 2.12. The ordering of the three is the ordering by how much of the sample's arrangement each one uses.

A detector built for the ordering

The best of three checks for a drifting scale fires at half the growth factor the standard one needs — 2.12 against 4.31 — and still leaves 6.50 points of coverage gone before it does, against 0.51 for serial correlation. The reversal was not a property of the test.

conformal · Exchangeability How often a Simpson reversal appears on the strongest baseline variable when its cut is slid along it, by the trial's size. The treatment helps every patient by the same six points. At 40, 80, 160, 320, 640 patients: some cut in the middle 80% reverses in 18.8%, 23.4%, 24.3%, 19.2%, 10.6% of trials; some decile in 12.2%, 13.2%, 14.1%, 11.0%, 6.2%; the median in 3.9%, 4.4%, 4.2%, 3.1%, 1.7%.

A cut that slides

Slide the cut on a trial's strongest baseline variable along every gap between observed values in its middle eighty per cent, and a trial of eighty whose treatment helps everyone equally shows a Simpson reversal somewhere in 23.4% of trials — against 13.2% for the nine deciles and 4.4% for the median. Sixty-five cut positions are worth about 9.5 independent cuts, and at 640 patients, with 513 positions, about 7.2: a table that moves one patient at a time is a continuous scan, and its count is set by the range scanned rather than by the number of cuts. The reversing cuts spread along the whole range, in 2.62 separate runs at eighty patients and 8.36 at 640.

reversal · Simpson

Named alongside it

The objects these essays reach for when they reach for this one.

Likelihood ratioSpecification searchSelection effectCritical valueMonte CarloStructural breakChange pointDegrees of freedomChi squaredCombinatorial searchDependenceIdentification

All concepts