The collection

Every essay — page 17

Essays 385 to 408 of 613, in the same order.

What a dictionary buys and what it costs

A balancing rule is a list of functions, and the list decides two things that pull against each other. What it protects against is decided by parity: an interaction between two odd functions is even, an odd function is orthogonal to an even one at every correlation, and the two things every trial balances — a mean and a median split — are both odd, so their worst case is exactly zero however many of them are held. What it costs is the assignments it leaves, and past a certain thinness those stop being one set: the admissible assignments split into an arrangement and its mirror image, no sequence of admissible single swaps joins them, and the walk that samples them is uniform on half the reference distribution for ever.

When a fixed width is reached

A fixed-width interval promises a precision rather than a sample size, so the trial stops when it has enough — and what the stopping rule is allowed to read decides whether the interval means anything. A rule that stops when its own interval is short enough is stopping on the spread it is about to quote, at a correlation of 0.938, and every weighting loses three to five points of coverage including the one told every true variance ratio. The width a set of weights will produce is predictable from the within-arm sums of squares alone, which are independent of every difference the interval is about; a rule that stops on the prediction is back at nominal, costs two and a half blocks, and gives up half of the width promise it was keeping.

The stopping rule costs more than the weighting does. Coverage over 2000 runs of a trial whose variance ratio drifts by a factor of twenty, at three ways of deciding when to stop. Twelve blocks fixed in advance is the top line and reproduces what a trial of fixed length delivers. Stopping when the reported interval is short enough is the bottom line, and it costs between 3.0% and 5.5% of coverage — including for the rule that is told every block's true ratio, which is what says the shortfall belongs to the stopping and not to the weights. Stopping on a width predicted from the within-arm sums of squares is the middle line, and it is back at the fixed-length values. The standard error on each point is 0.49%.

A width the trial has to stop for

The weighting that covers at 94.9% on twelve blocks covers at 91.5% when the trial stops as soon as its interval is short enough — and so does the rule that is told every block's true variance ratio. The shortfall is the stopping, not the weights.

4 figures · Stopping, part 10
Two promises, and no rule here keeps both. A fixed-width procedure promises two things: that the interval covers at its nominal rate, and that it is no wider than the width asked for. Over 1500 runs of the modelled weighting, a rule that stops when the interval it will report is short enough keeps the width — only 2.0% of runs come out wider than 0.34 — and covers at 91.13% against a nominal 95%. A rule that stops on a width predicted from the within-arm sums of squares covers at 94.80% and comes out wider than promised on 42.3% of runs. The two promises are in conflict because keeping the second one exactly requires conditioning on the very quantity that has to be independent of the stopping time for the first.

Stopping on the arms

The width a trial will report is predictable from quantities the interval is not about. A rule that stops on the prediction covers at 94.5% where one that stops on the interval covers at 91.5, and it costs two blocks and half of the width promise.

4 figures · Width, part 8
What a fixed-width interval covers, by the number of blocks the trial ran before it stopped. Two thousand runs of each rule, the modelled weighting, a promise of 0.34. Reading its report: 4–8 blocks, 22.3% of runs, 78.2%; 9–12 blocks, 16.6% of runs, 90.4%; 13–16 blocks, 18.4% of runs, 96.2%; 17–20 blocks, 17.4% of runs, 96.0%; 21–28 blocks, 17.9% of runs, 96.4%; 29–36 blocks, 7.4% of runs, 99.3% — 91.45% overall. Reading the arms: 4–8 blocks, 0.0%, none; 9–12 blocks, 0.9%, 94.4%; 13–16 blocks, 30.4%, 95.6%; 17–20 blocks, 50.0%, 93.9%; 21–28 blocks, 18.0%, 94.4%; 29–36 blocks, 0.7%, 92.3% — 94.50% overall.

The trials that stopped early

A fixed-width trial that stops when its own interval is short enough covers 91.45% — an average of 78.2% among the 22.3% of runs that stop within eight blocks and 96% to 99% among those that run longer. Widening every interval by 17.1% brings the average to 95% and leaves the early stops at 85.6%, while 92.8% of runs now report an interval wider than the width they promised. Even doubling every interval leaves the early stops short.

6 figures · Width, part 9
The fixed-width trial's coverage when the outcomes are not normal, for both stopping rules. normal: stopping on the arms 94.05% after 18.1 blocks, on the report 89.95%; log-normal, skewness 0.95: stopping on the arms 94.70% after 18.5 blocks, on the report 90.80%; log-normal, skewness 2.26: stopping on the arms 94.15% after 19.3 blocks, on the report 90.25%; log-normal, skewness 4.75: stopping on the arms 94.45% after 18.7 blocks, on the report 90.50%; t, five degrees of freedom: stopping on the arms 94.35% after 18.3 blocks, on the report 90.30%; skewness 4.75, arm A only: stopping on the arms 93.80% after 26.0 blocks, on the report 89.90%; skewness 4.75, arm B only: stopping on the arms 93.60% after 14.2 blocks, on the report 89.95%; equal variances, normal: stopping on the arms 94.75% after 11.4 blocks, on the report 90.90%; equal variances, skewness 4.75: stopping on the arms 94.05% after 11.1 blocks, on the report 92.00%.

A width rule on skewed outcomes

The blinded fixed-width rule rests on a within-arm spread being independent of the arm means, which only normal samples guarantee. On outcomes with a skewness of 4.75 the independence fails and the overall coverage barely notices — 93.60% to 94.70% across every shape counted, against 94.05% on normal outcomes. What skew moves is the runs that stop by twelve blocks, which cover about 90% with the skew in one arm, and the trial's length: a variance ratio corrected on normal theory lengthens it from 18.1 blocks to 26.0 with the skew in the first arm and shortens it to 14.2 with the skew in the second.

6 figures · Width, part 10
A lean, its reflection, and none. The mean error of each weighting's estimated difference at a fixed eighteen blocks, 2,000 trials a shape, with two standard errors. normal: the modelled ratio, every block 0.0031, the modelled ratio, each block left out 0.0028, the modelled ratio, cross-fitted halves 0.0023, the true ratios 0.0037; skewness 0.95: the modelled ratio, every block 0.0092, the modelled ratio, each block left out -0.0134, the modelled ratio, cross-fitted halves -0.0014, the true ratios 0.0002; skewness 2.26: the modelled ratio, every block 0.0166, the modelled ratio, each block left out -0.0298, the modelled ratio, cross-fitted halves -0.0042, the true ratios -0.0029; skewness 4.75: the modelled ratio, every block 0.0255, the modelled ratio, each block left out -0.0455, the modelled ratio, cross-fitted halves -0.0055, the true ratios -0.0051.

A weighting that does not read its own spreads

On skewed outcomes a block's spreads move with its means, so a weighting fitted to every block's spreads leans: +0.0255 at a skewness of 4.75. The obvious repair — fit each block's weight with that block left out — does not remove the lean. It reverses it and nearly doubles it, to −0.0455, because a weighted mean divides by the sum of the weights and every other block's weight still reads the one left out. Splitting the trial into two halves and weighting each by the other's fit does remove it: −0.0055, indistinguishable from the weighting told the true ratios, for a spread about a tenth wider.

4 figures · Width, part 11

Fitted together, or fitted after

Every whitening in this collection is a two-step rule: fit a candidate, read the dependence off what is left over, whiten, fit again. The step nobody examined is the first one. A fit removes memory as well as signal, and how much is arithmetic rather than noise — the residuals of a hundred and twenty rows report a lag-one coefficient of 0.73 where the errors report 0.78 and the law says 0.80. Estimating the coefficient with the line instead of after it recovers most of that, and iterating the two-step rule to its fixed point recovers nearly as much — because a fixed point of the sum of squares is not a maximum of the likelihood, and the difference between them is real, one-sided, and worth nothing at all in the decision the number feeds. The order the same criterion picks moves by nearly two, which is worth a great deal more.

The dependence, at four removes. Under AR(1) at 0.8, four different sequences all called the dependence. The top line is the law. The middle line is what a sample of 120 errors reports on average — computable exactly, because the expectation of a sample autocovariance is arithmetic once the covariance is known. The lower line is what a candidate's residuals report, which is what every two-step rule in this collection actually reads: a fit removes variance, and it removes more of the persistent part than of the rest. At the first lag the three are 0.800, 0.7773 and 0.7338. The dots are counted from draws and share no arithmetic with the line they sit on; the worst departure is 1.2 standard errors.

A dependence fitted with the line

Every whitening in this collection reads the dependence off a set of residuals, and residuals are not errors. Fitting the two together recovers most of what that costs, and changes almost nothing about the decision it feeds.

6 figures · Dependence, part 8
How much memory a fit takes out, candidate by candidate. Under AR(1) at 0.8, the lag-one autocorrelation a candidate's residuals report, computed exactly for each candidate on 200 draws. The upper line is the law at 0.8000. A candidate that is an intercept alone reports 0.7773 — which is exactly what a sample of 120 errors reports, because an intercept annihilates the sample mean and nothing else, and the two arithmetics agree to the last bit. Every predictor after that takes more out, down to 0.7341 at the fullest candidate. That is the collision this field is about: the rule every whitening here uses estimates its nuisance once, from the fullest candidate, so that the criteria stay comparable — and the fullest candidate is the one whose residuals report the least.

The fit that takes the memory out

A candidate's residuals report less dependence than its errors do, and how much less is arithmetic rather than noise. The rule used for a good reason reads the series that has lost the most.

3 figures · Dependence, part 9
One likelihood, three answers. The concentrated Gaussian log-likelihood of one sample of 120 rows under AR(1) at 0.8, as a function of the correlation the errors are whitened at. Three rules put three different numbers on this curve. The two-step rule reads the least-squares residuals and lands at 0.7616, giving up 0.304 of log-likelihood. Iterating moves it to 0.8080 and gives up 0.002. The maximum is at 0.8044. The curve is not flat between them: what a fixed point of the residual update finds is a solution of a different equation, and the difference is the Jacobian term ½log(1 − ρ²), which grows as the correlation does.

Iterating is not maximising

Re-reading a correlation from the generalised residuals and refitting converges in seven steps. What it converges to solves the first-order condition of a sum of squares, and the likelihood has one term more than that.

5 figures · Dependence, part 10
One window for the table, or one each. The regret of the same fifteen-candidate table under AR(1) at 0.8 over 150 draws, with the window attached three ways. Chosen once from the fullest candidate's residuals it gives up 0.02518. Chosen from each candidate's own residuals, with the covariance estimate still shared, it gives up 0.02799 — a paired cost of 0.00281 at 2.0 standard errors for the tuning parameter alone. Estimating the covariance per candidate as well costs 0.01087, so the objection already on record is about 3.9 times the size of the one that was not.

A window for every candidate

The window and the order a whitening needs are chosen once, from the fullest candidate, on an argument that was made about an estimated covariance. A tuning parameter is not a covariance, and the two cost different amounts.

3 figures · Order selection, part 8
What iterating the band is worth. How much lower the prediction error is when the Bartlett band is iterated to its fixed point than when it is read once off the least-squares residuals, paired on the draw, with two standard errors, 300 draws at 120 rows for each law and window. First-order autoregression: 4 lags 2.84 ± 0.36, 8 lags 3.43 ± 0.58, 20 lags 4.16 ± 0.80; five-period moving average: 4 lags 2.04 ± 0.28, 8 lags 2.69 ± 0.42, 20 lags 4.32 ± 0.68; long memory: 4 lags 0.83 ± 0.14, 8 lags 1.13 ± 0.25, 20 lags 1.33 ± 0.44 thousandths. Every one is below zero by more than two and a half standard errors; the gain is largest at the widest band under the two short-memory laws.

A fixed point with no likelihood under it

A tapered band of autocovariances has no likelihood to maximise, so the one rule that can fit its dependence together with the regression is to iterate: read the band off the residuals, refit, read it again. It converges on every draw, in seven to eleven fits at the median, gives back about half the memory the first fit took out of the residuals under a short-memory law, and lowers prediction error by two to four thousandths — five to eight paired standard errors. Under the law a first-order likelihood describes, its maximum still wins; under the two it does not, the band's fixed point beats it.

5 figures · Dependence, part 11

Paying for a search

A two-regime whitening chooses its change point by searching a profile, and then reads a criterion that counts parameters. Nothing charges for the search. Nothing can, in the usual way: under no break the two regimes share a coefficient and the break point is not identified at all, so the likelihood ratio is a supremum over a parameter that exists only under the alternative and no count of restrictions describes its distribution. Searching a hundred and twenty rows for a change point that is not there manufactures about five units of ratio where one parameter costs two — and a rule that counts the break as free splits a stationary sample on 99% of draws. A second break costs as much again, on a sample that has only ever had one.

The profile a break point is chosen from. One sample of 120 rows under a break in the persistence, fitted as two first-order regimes at every admissible break point. The maximum is at row 78, where the true break is at 60. The shaded band is every break point within two log-likelihood units of the best one — 8 of the 73 positions searched, which is 11% of the range. The horizontal line is the one-regime fit the search is compared against; the whole profile is above it, at every position, which is the point: a maximum over 73 candidates is above the null by construction and not by evidence.

A break that was looked for

A two-regime whitening finds its change point by maximising a profile, and then reads a criterion that counts parameters. Under no break there is no parameter to count, because every position describes the same model.

6 figures · Break point, part 1
What a search manufactures, law by law. The average likelihood ratio a search over 120 rows reports, on each of the four laws, over 400 draws. Three of them have no break at all and report 5.080, 4.839 and 4.748; the fourth has one and reports 8.757, so the real break is worth only 3.677 beyond what the search would have found anyway. The dashed line is 2, which is what an information criterion charges for one extra parameter. A search costs about two and a half of them, and the number is a measurement rather than a count.

What a search costs in parameters

An information criterion's penalty is an estimate of the optimism a fit carries. For a break point the optimism can be measured and cannot be counted, and it comes to about two and a half parameters.

3 figures · Break point, part 2
How often the split is taken, and by which rule. Over 400 draws on each of five laws. The first two rows have no break in them at all, the last two have one at row 60, and the middle one is a moving average. A criterion that counts a fitted two-regime model's parameters and nothing else takes the split on 99% of draws where there is no break. Counting the break point as one more parameter brings that to 67%. Charging what the search actually manufactures — 5.16 units, measured on a law with no break — brings it to 16%, and still takes the split on 61% of draws where there is one.

Choosing whether to break

Charging what the search manufactures takes a rule from splitting a stationary sample on 99% of draws to 16%. It also costs regret, because the two mistakes a rule can make are not the same size.

3 figures · Break point, part 3
What a second break adds. Over 200 draws, the likelihood ratio a search over one break point reports, and how much more a search over an ordered pair adds on top of it. Under AR(1) at 0.8, which has no break at all, the first search manufactures 5.697 and the second adds 4.278. Under a law with exactly one break — where a second one is as absent as the first was in the row above — the first search reports 9.442 and the second still adds 5.800. Searching for something that is not there costs the same whether or not something else was there to find.

A second break on a flat profile

Searching a hundred and twenty rows for one change point manufactures five units of likelihood. Searching for a second manufactures four more, on a series that has at most one — and on a profile whose whole range is under seven.

3 figures · Break point, part 4
Where two profile likelihoods put the break. Under a break in the persistence at row 60 of 120, and the same break turned over, over 300 draws: how often each search's estimate lands within twelve rows of the truth, with its average and spread beside it. persistent first · one scale for the whole series: 46.7%, at 74.8 with a spread of 11.8; persistent first · a scale per regime: 94.3%, at 61.8 with a spread of 5.8; persistent last · one scale for the whole series: 49.3%, at 45.3 with a spread of 11.7; persistent last · a scale per regime: 93.0%, at 56.7 with a spread of 5.9.

The break a shared scale hid

The break searches in this field all maximised a likelihood with one innovation variance for the whole series, and on a break from 0.95 to 0.65 they found it badly: an average of 74.8 where the truth is 60, within twelve rows on 47% of draws, from a profile everyone called flat. Give each regime its own innovation variance — one more parameter, and the one the law actually has — and the same search lands at 61.8 with a spread of 5.8, within twelve rows on 94% of draws, reporting four times the likelihood. The break was never barely there; the likelihood was blind to the half of its evidence that is in the scale. Under no break the extra parameter costs about one and a half units of manufactured likelihood.

3 figures · Break point, part 5

What a chain cannot report

A thin admissible set falls into an arrangement and its mirror image, and the walk that samples it is uniform on half the reference distribution for ever while every diagnostic passes. That was found by enumerating fourteen units, and enumeration stops at about twenty-four. The test that does not enumerate is two chains — one from an assignment, one from its complement — compared on a statistic that changes sign under the complement, with a third chain from the same starting point to say whether a large reading is a fact about the set or about the length of the run. It agrees with the enumeration at every tolerance where the answer is known. At two hundred units it reports something else: the walk reaches the whole set at the tolerances a trial would use, and past a point the diagnostic stops agreeing with itself.

The test, checked where the answer is known. A fourteen-unit trial at eight tolerances. At each one the admissible set is enumerated — 1534, 886, 304, 158, 126, 116, 102, 84 assignments — and its components counted, which is only possible because 3432 equal splits of fourteen units can be walked. The dots are the test, which walks none of them: two chains, one started at an assignment and one at its complement, compared on a statistic the rule was not handed. Filled marks are tolerances the enumeration says leave the set in more than one piece. The test fires on every one of them and on none of the others, 0 misses and 0 false alarms.

A test rather than a survey

A thin admissible set falls into an arrangement and its mirror image, and the walk that samples it is uniform on half the reference distribution for ever. That was found by enumerating fourteen units, and enumeration stops at twenty-four.

4 figures · Randomisation, part 11
Three readings, one verdict. Every tolerance of a fourteen-unit trial, with three comparisons on each. The first is between a chain started at an assignment and a chain started at its complement, which is what a mirror split separates. The second is between two chains started at the same assignment on different streams, which nothing about the set can separate — so a large reading there says the run is too short and not that the set is in pieces. The third is the same comparison on the statistic's absolute value, which is symmetric under the complement and therefore blind to the split by construction. The verdict is the pattern rather than any one line: the split is called only where the first fires and the other two do not, which happens at exactly the tolerances the enumeration calls disconnected — 0.8, 0.75, 0.7.

The statistic that changes sign

A test for an unreachable half needs a quantity that tells one half from the other. Every symmetric reading of a mirror pair is identical, and a magnitude is the natural thing to reach for.

4 figures · Randomisation, part 12
Thinness stays put and reachability does not. The same balancing rule and the same tolerance at five trial sizes. The admitted share barely moves — one admissible assignment in 27, 30, 30, 20, 18 — because the acceptance rate of a rerandomisation is a fact about the basis rather than about the number of units. What does move is the number of single swaps available: 36, 49, 64, 81, 100, growing like a quarter of the square of the trial size. Filled marks are sizes whose admissible set falls into more than one piece. The set is in 4 pieces at 12 units, 2 at 14, and one piece from 16 upwards. The fourteen-unit result is a statement about fourteen units.

A defect that is about size

The admitted share of a rerandomisation barely moves with the number of units. The number of admissible neighbours grows like the square of it, and that is what decides whether the walk can go everywhere.

4 figures · Randomisation, part 13
What the diagnostic says at two hundred units. The same test run 8 times on independent streams, at five tolerances of a two-hundred-unit trial, 40,000 steps each. At the loosest tolerance every run says the same thing — the walk reaches the whole set — and it keeps saying it as the set is thinned. Past a point the runs stop agreeing with each other: at the tightest tolerance here 6 of 8 report that the chains have not mixed and 2 report a split, which is a diagnostic disagreeing with itself rather than a property of the set. That disagreement is the honest answer at this size, and it is one nothing in this collection could give before: an enumeration stops at about twenty-four units.

The diagnostic at two hundred

Pointed at a trial size no enumeration reaches, the test gives three answers rather than one — and past a certain thinness it stops agreeing with itself, which is the honest reading and the one nothing could give before.

4 figures · Randomisation, part 14
When each unit first changed arm. One design of two hundred units, the rule balancing the covariate, its square and a median cut, and a swap-one-pair chain run for 1.6 million steps from a random admissible start, at six tolerances: for each unit, the step at which its arm first differed from the start, the two hundred sorted from first to last. At 1: half the units had moved by step 79, nine in ten by 269, all by 1,073; At 0.6: half the units had moved by step 79, nine in ten by 362, all by 1,073; At 0.4: half the units had moved by step 106, nine in ten by 429, all by 1,593; At 0.3: half the units had moved by step 153, nine in ten by 645, all by 3,878; At 0.22: half the units had moved by step 194, nine in ten by 747, all by 1.1 million; At 0.16: half the units had moved by step 237, nine in ten by 1270, and 1 never moved. The last units to move set the run length a diagnostic built on them needs, and at the tightest tolerance one never does.

A unit that has not moved

At two hundred units the two-chain diagnostic could not settle its own verdict, because its standard error needed a relaxation time the run was too short to estimate. Reading which units moved needs no standard error. Run for 1.6 million steps on five designs, it finds a unit in place in both chains on three designs at the tightest tolerance and one at the next — and every one of them is a unit no swap can move from any admissible assignment, by arithmetic. The admissible set does split at two hundred units, and the split is one extreme unit.

4 figures · Randomisation, part 15

A guarantee that needed a symmetry

What a balancing rule can remove of an interaction is decided by parity, and parity is a statement about a symmetry of the law rather than about the covariate. Keep the dependence and change the marginals — a Gaussian copula under a monotone transformation — and the two things every trial balances come apart. A median split is a function of the sign of the latent normal whatever the marginal is, so its exact zero survives every transformation to the last digit. A mean is odd only when the marginal is symmetric, and its zero is gone at a skewness of one. The separating case is a marginal that is heavy-tailed and symmetric, where every zero holds exactly: it is not normality the guarantees needed. And a threshold at a value on the covariate's own scale never had one at all.

FieldsThreadsSeriesConceptsFigure librarySearch