Normal Distribution
nothing is called 95% until it has been counted
Concept

Moving average — where it appears

A series formed by averaging a fixed number of independent shocks, so that its autocorrelation is exactly zero past that many lags. It is the natural opposite of an autoregression, whose autocorrelation never reaches zero, and no autoregression of finite order reproduces one exactly.

Named by 11 essays across 7 fields — each of them below, with the objects they name alongside it.

Four dependences a single parameter cannot tell apart. Every law here is standardised to a lag-one autocorrelation of 0.8, so a rule told the errors are a first-order autoregression finds the same number in all four and has no way of seeing what separates them. The geometric decay is the world in which estimating a covariance rather than naming it was priced, and found to cost. The five-period moving average has 0.200 at the fourth lag and exactly nothing past it, where the geometric law says 0.328 at the fifth. Long memory at d = 4/9 is still at 0.576 by the twentieth lag, where the geometric law has reached 0.012. The break has no autocorrelation function at all: what is drawn for it is the average over the pairs at each gap, which is what a stationary estimate converges to.

A dependence with a shape

Four ways for errors to repeat, all with the same first lag and nothing else in common. A rule told the errors are a first-order autoregression finds the same number in all four, and is right about one of them.

general · Dependence The window a whitening wants is not the memory of the errors. Regret under a five-period moving average as the tapered estimate is given more lags, over 120 draws at n = 120. The best window is L = 30; the automatic bandwidth is 4 and the error model's own likelihood chooses 9.7 on average. Both land in the same place and both are short, and the reason is the taper: a Bartlett weight at lag k is 1 − k/(L + 1), so a window of 8 keeps 0.556 of whatever the fourth lag carries and a window of 30 keeps 0.871. A window has to be several times the memory before it stops removing the memory. The dashed line is the rule told the errors are a first-order autoregression, which needs no window at all.

The window a whitening wants

Every law here is best whitened by a window several times longer than its own memory, including the one whose memory ends at the fourth lag. The three ways of choosing it from the sample all land in the same place, and it is the wrong one.

general · Order selection Four sequences, and the rule only ever sees the last one. Under long memory at d = 4/9, four things that are all called the dependence. The law itself is the top line. What a sample of 120 rows reports on average is the second, computed exactly: subtracting a sample mean takes the first lag from 0.800 to 0.538. What a candidate's residuals report is the third, lower again at 0.472, because a fit removes dependence along with signal. The autoregressions are fitted to that third sequence and reproduce it exactly out to their own order — the Yule–Walker equations are solved to make it so — so everything they say past that is extrapolation. At the twentieth lag the law has 0.576, the residuals report 0.006, and an AR(8) extrapolates 0.028.

The order the tail is drawn at

A fitted autoregression reproduces the sample exactly at the lags it was fitted on, so everything it says past them is extrapolation — and the order is the dial that decides how much of it there is.

general · Order selection A premium that shrinks and a payout that grows. The relative error of the trapezoid's implied variance with the block length read off the whole autocovariance sequence, minus its error with the length from the lag-one correlation alone, in points, at four sample sizes and 400 draws each. Under the first-order autoregression the plug-in's algebra was written for, reading the sequence costs 3.87 at 120, 2.37 at 240, 2.48 at 480, 1.15 at 960. Under the five-period moving average, which the plug-in reads as an autoregression at 0.8, it is 0.43 at 120, -1.72 at 240, -3.01 at 480, -3.77 at 960 — a tie at 120 rows and a gain that grows with the sample.

A length read off the whole sequence

The automatic block length reads one number off the sample — the lag-one correlation — and assumes the rest of the dependence is geometric. Reading the whole autocovariance sequence instead removes the assumption and puts the length where the law wants it. At a hundred and twenty rows it buys nothing, even where the assumption is wrong; it costs 3.87 points where the assumption is right, and by 960 rows that premium is 1.15 and the payout under a law the plug-in misreads is 3.77. Under long memory neither rule, nor the best length on each draw, gets within ninety per cent.

feasible · Bootstrap What four length rules cover, 120 rows. The coverage of the t interval on the overlapping scale at 120 rows, when the block length is the shortest one predicted to reach 94.5% under each of four readings of the share, over 4,000 draws of each law. AR(1) at 0.7: geometric 96.2%, read to the chosen lag 93.2%, read to lag 6 93.7%, known 96.5%. a five-period moving average: geometric 96.5%, read to the chosen lag 93.5%, read to lag 6 94.5%, known 95.3%. a break, 0.95 then 0.65: geometric 95.1%, read to the chosen lag 93.0%, read to lag 6 92.5%, known 95.2%. the break turned over: geometric 94.8%, read to the chosen lag 92.5%, read to lag 6 92.2%, known 95.0%.

A share read off the sample

A block length chosen for an interval's coverage needs the share of the mean's variance the scale will find, and the published rule reads it from a first-order autoregression at the sample's lag-one value. Reading it from the sample's own autocovariances instead was supposed to survive series nobody had modelled. At 120 rows it covers 92.5% to 93.5% on all four laws — worse than the wrong shape on every one — because a centred series' autocovariances sum to exactly zero and the share they imply is too generous; read to lag forty it covers 84%. The assumed shape costs width, not coverage: 17% too wide where dependence stops at lag four, and nothing visible under a break, where the interval covers for a reason the share knows nothing about.

taper · Bootstrap The error of three estimates against the length of the series, AR(1) at 0.7. Under AR(1) at 0.7, the root-mean-square error of three estimates of the long-run variance, as a share of the truth, at 120, 240, 480, 960 rows. a first-order autoregression: 45.5%, 33.3%, 23.1%, 15.8%; an autoregression, order by AIC: 50.0%, 39.3%, 26.3%, 18.7%; the trapezoid block, best length: 42.4%, 32.9%, 25.4%, 19.5%.

A model against a window

A block resample's long-run variance is attenuated by its window, and the construction that escapes the window is a model: fit an autoregression and sum its dependence to every lag in closed form. Measured on both halves of the answer, the escape is narrower than it looks. Correctly specified, on 120 rows of a first-order autoregression, the model's error is 45.5% of the truth against the trapezoid block's 42.4% — it removes the bias and buys a spread larger than the bias was — and it pulls ahead only from about 480 rows. Wrongly specified, on a five-period moving average, it is off by 84% to 105% at every length, and even an order chosen by the criterion trails the window throughout.

crossing · Bootstrap The same dial, on a list that steps by one. The probability that a per-candidate tuning list changes which candidate the table selects, at each of 5 separations, for both tuning parameters at a matched list length of 8, with errors from a five-period moving average. The sieve order runs from 29.5% to 5.8%, a factor of 5.13; the whitening window, from 26.3% to 4.8%, a factor of 5.53. What is held is the number of options, the table, the law, the sample size and the seeds; what cannot be held is the size of a step, since an integer step and a geometric step are different amounts of change. The dial moves both, and it moves them by 1.08 times as much on one as on the other.

The same dial under a second law

On a first-order autoregression, separating the candidates moved the chance that a tuning list changes the winner 2.35 times as far on a whitening window as on a sieve order, and the difference was put down to the step: an integer step is worth a third of a geometric one. Rerun the whole sweep with errors from a five-period moving average and the difference is gone — the probability spans a factor of 5.53 on the window and 5.13 on the order, a ratio of 1.08 — and near the null both lists leave about five and a half candidates within reach, where the autoregression left 3.6 and 1.9. The two factors still point opposite ways under both laws. The step's size was a property of the law the lists were tuned on.

turnover · Order selection Five length rules, read on two intervals. The coverage of a 95% interval for the mean at the block length each of five rules picks, on the percentile interval and the studentised one built from the same resamples, under a five-period moving average and a first-order autoregression at 120 and 480 rows, 2,000 draws each. moving average at 120: percentile 88.5%, 85.3%, 86.4%, 86.3%, 88.8%; studentised 92.5%, 87.9%, 94.9%, 92.6%, 95.0%. moving average at 480: percentile 90.5%, 90.5%, 90.4%, 91.6%, 91.9%; studentised 91.2%, 91.2%, 94.5%, 93.3%, 95.0%. autoregression at 120: percentile 84.7%, 80.2%, 85.8%, 85.4%, 85.9%; studentised 90.3%, 83.0%, 92.0%, 90.9%, 94.3%. autoregression at 480: percentile 87.0%, 87.0%, 90.0%, 90.5%, 90.9%; studentised 88.6%, 88.6%, 93.0%, 93.0%, 94.2% (in the order eight, from the protocol, n to the one third, the plug-in, the whole sequence, the best on the draw). The rules spread over at most 5.6% on the percentile interval; the interval moves the same rule by up to 8.5%.

A ranking that belongs to the interval

Five ways of choosing a block length, read on the coverage of the interval each length builds rather than on the variance it implies. On the percentile interval the variance's verdict mostly survives: under a five-period moving average the plug-in and the whole-sequence rule tie at 120 rows and the sequence rule leads at 480. Build a studentised interval from the same resamples and the lead reverses — the plug-in's longer blocks cover 94.5% against 93.3% — because an interval that widens for its own uncertainty is paid for a long block where a percentile interval is punished for one. Under long memory no length and no interval covers two times in three.

feasible · Bootstrap One order's interaction climbs under long memory. The break-and-window pair's interaction — the joint search's quantile less the sequential run's — at five quantile levels, in the field's order (break first, solid) and the other (window first, dashed), under three laws, 1200 draws each. first-order autoregression: break first 1.57, 1.29, 1.19, 3.22, 2.75; window first 0.98, 1.26, 1.53, 2.55, 2.14. five-period moving average: break first 2.00, 1.88, 2.29, 2.63, 0.60; window first 0.16, 0.21, 0.26, 0.29, 0.00. long memory: break first 0.53, 0.80, 1.05, 0.46, 1.35; window first 2.01, 1.62, 4.75, 6.07, 7.85, at the 50%, 75%, 90%, 95% and 99% points. Under two of the laws, and in the field's order under all three, the interaction stays within a few units of its median. Under long memory with the window searched first it climbs from 2.01 to 7.85, against an average of 1.99; the same order's averages under the other two laws are 0.94 and 0.17.

A tail the law gives the pair

Split the over-charge of two searches at their critical value and the interaction — the configurations only the joint search reaches — stays near its average while the overlap climbs. That held under one law. Under a five-period moving average and under long memory it holds for every pair in the order the field reads, to within three units. In the other order, one pair acquires a tail of its own: with the window searched first and the break second, the interaction under long memory climbs from 2.0 at the median to 7.9 at the 99th percentile, four times its average.

separate · Break point What iterating the band is worth. How much lower the prediction error is when the Bartlett band is iterated to its fixed point than when it is read once off the least-squares residuals, paired on the draw, with two standard errors, 300 draws at 120 rows for each law and window. First-order autoregression: 4 lags 2.84 ± 0.36, 8 lags 3.43 ± 0.58, 20 lags 4.16 ± 0.80; five-period moving average: 4 lags 2.04 ± 0.28, 8 lags 2.69 ± 0.42, 20 lags 4.32 ± 0.68; long memory: 4 lags 0.83 ± 0.14, 8 lags 1.13 ± 0.25, 20 lags 1.33 ± 0.44 thousandths. Every one is below zero by more than two and a half standard errors; the gain is largest at the widest band under the two short-memory laws.

A fixed point with no likelihood under it

A tapered band of autocovariances has no likelihood to maximise, so the one rule that can fit its dependence together with the regression is to iterate: read the band off the residuals, refit, read it again. It converges on every draw, in seven to eleven fits at the median, gives back about half the memory the first fit took out of the residuals under a short-memory law, and lowers prediction error by two to four thousandths — five to eight paired standard errors. Under the law a first-order likelihood describes, its maximum still wins; under the two it does not, the band's fixed point beats it.

together · Dependence The cut-off lag, before and after the correction. Coverage at 120 rows when the block length is chosen from a share read off the sample's autocovariances to a stated lag, as read and corrected for the centring. first-order autoregression, as read: 93.9% at lag 6, 93.3% at lag 12, 88.7% at lag 24, 84.5% at lag 40. the same, corrected: 94.0% at lag 6, 94.5% at lag 12, 91.9% at lag 24, 89.2% at lag 40. moving average, as read: 94.5% at lag 6, 92.3% at lag 12, 88.7% at lag 24, 85.0% at lag 40. the same, corrected: 94.7% at lag 6, 93.7% at lag 12, 91.7% at lag 24, 89.8% at lag 40. a break, as read: 92.1% at lag 6, 93.1% at lag 12, 90.1% at lag 24, 83.5% at lag 40. the same, corrected: 92.2% at lag 6, 93.8% at lag 12, 92.2% at lag 24, 89.7% at lag 40.

A sequence that knows it was centred

A block length read off a sample's own autocovariances gets worse the further out it reads, because a centred series' autocovariances sum to exactly zero. To first order every lag loses the same amount, about the long-run variance over the sample size, and that can be added back. Corrected, a reading to lag forty goes from 84.5% to 89.2%, and to lag twelve from 93.3% to 94.5%. But the correction overshoots on the long lags, and on the rule that picks its own cut-off, which is short, it moves coverage by three tenths of a point. The identity was the reason the lag mattered. It was not the reason the rule falls short.

taper · Bootstrap

Named alongside it

The objects these essays reach for when they reach for this one.

Long memoryMonte CarloAutocorrelationAutoregressionBlock lengthLong-run varianceRegretSample autocovarianceBlock bootstrapDependenceGeneralised least squaresInformation criterion

All concepts