Concept

Autoregression — where it appears

A model in which each observation is a weighted sum of the ones before it plus a fresh shock, so its autocorrelation continues at every lag rather than stopping. Fitting one to a series gives a rule for continuing the dependence past the lags the fit was told about.

Named by 18 essays across 11 fields — each of them below, with the objects they name alongside it.

Four dependences a single parameter cannot tell apart. Every law here is standardised to a lag-one autocorrelation of 0.8, so a rule told the errors are a first-order autoregression finds the same number in all four and has no way of seeing what separates them. The geometric decay is the world in which estimating a covariance rather than naming it was priced, and found to cost. The five-period moving average has 0.200 at the fourth lag and exactly nothing past it, where the geometric law says 0.328 at the fifth. Long memory at d = 4/9 is still at 0.576 by the twentieth lag, where the geometric law has reached 0.012. The break has no autocorrelation function at all: what is drawn for it is the average over the pairs at each gap, which is what a stationary estimate converges to.

A dependence with a shape

Four ways for errors to repeat, all with the same first lag and nothing else in common. A rule told the errors are a first-order autoregression finds the same number in all four, and is right about one of them.

general · Dependence

The number the comparison was missing. What it costs to choose the tuning parameter for every candidate separately rather than once for the table, under AR(1) at 0.8, paired on the draw. The window's figure is the one the earlier field reported; the order's is the one it named and did not make. They are the same size — 0.00401 against 0.00360, at 2.30 and 1.72 paired standard errors — and matching the lists at eight values leaves them the same size again. The prediction that the longer list would make the order's cost the larger of the two is not what happens; what happens is that the two rules cost the same once they are scored by the same criterion, which took a missing term to arrange.

The comparison that was not made

Choosing a whitening's window separately for every candidate costs 0.00401 of regret. The same question about an order was named and left, because the two lists are different lengths. The order's answer is 0.00360, and matching the lists changes almost nothing.

lists · Order selection

The term that cancels, and the term that does not. The volume each candidate's whitening moves — log|Ω̂| — for a sieve of order 4 on one sample of 120 rows. Estimated once from the fullest candidate and used for the whole table, it is the same number for every candidate, so it drops out of every difference the criterion reads: that is why nothing in this collection has ever needed to carry it. Estimated from each candidate's own residuals it ranges over 23.87, which is more than a parameter is worth, and the criteria being compared are then fits made under different error models with no term saying so. The window's rule has carried this term since the estimated-covariance field and the sieve's never had it.

The volume a whitening moves

A sieve's whitening has a determinant and this collection's criterion for it never carried one. Shared across a table the term cancels exactly, which is why nothing ever noticed; used per candidate it is worth more than a parameter and the whole comparison turns on it.

lists · Criterion

What a longer list actually changes. How often the fifteen candidates choose different tuning parameters, at a true null where every one of them contains the truth, so a disagreement is manufactured rather than discovered. The order's list is an interval of integers, and thinning it moves the rate smoothly from 0% at two values to 39% at thirteen. The window's is not an interval — it runs 0, 1, 2, 4, 8, 12, 20, 30 — so a thinned window list jumps depending on whether it happens to keep the width the criterion wants, between 0% and 42% with no order to it. So "the same length" was never quite the same thing for the two rules, and it is a smaller effect than the field it was invoked to explain.

A list is not a rule

How often fifteen candidates disagree about a tuning parameter runs from nothing at two values on the list to two draws in five at thirteen. What the disagreement costs does not move at all.

lists · Order selection

Four sequences, and the rule only ever sees the last one. Under long memory at d = 4/9, four things that are all called the dependence. The law itself is the top line. What a sample of 120 rows reports on average is the second, computed exactly: subtracting a sample mean takes the first lag from 0.800 to 0.538. What a candidate's residuals report is the third, lower again at 0.472, because a fit removes dependence along with signal. The autoregressions are fitted to that third sequence and reproduce it exactly out to their own order — the Yule–Walker equations are solved to make it so — so everything they say past that is extrapolation. At the twentieth lag the law has 0.576, the residuals report 0.006, and an AR(8) extrapolates 0.028.

The order the tail is drawn at

A fitted autoregression reproduces the sample exactly at the lags it was fitted on, so everything it says past them is extrapolation — and the order is the dial that decides how much of it there is.

general · Order selection

Every pair of sieve orders. Each cell is a tuning list of exactly two sieve orders, the row's and the column's, and its shade is how often fifteen candidate regressions choose different values from it, at a true null over 240 draws. Every pair holding order zero disagrees on no draw at all, because the criterion never picks zero under this law. Every adjacent pair of positive orders disagrees on between 17% and 25% of draws — 22.1% for one and two, 25.0% for eleven and twelve — and the rate falls as the two orders move apart, to 5.0% for one and twelve. The largest cell is 25.0%.

Two values the criterion cannot tell apart

Fifteen candidates disagree about a whitening's order on two draws in five when the list holds all thirteen orders — and on more than one in five when it holds only one and two. Every adjacent pair of positive orders disagrees about as often as that pair does, and every pair holding an order the criterion never picks disagrees on none. A list's disagreement is set by the pairs of values on it that the criterion cannot tell apart, not by how many values it has, and one candidate's own doubt about its pick predicts all fifteen candidates' disagreement with a rank correlation of 0.96.

lists · Order selection

What a longer sample settles

At a hundred and twenty rows, letting each candidate choose its own whitening changes which of fifteen candidates wins on about an eighth of draws, whatever the list, the table or the law. At four hundred and eighty rows it changes the winner on 1.0% of draws for a window and 2.3% for an order, and at nine hundred and sixty on a quarter of one per cent. The sample settles it through one factor only: how often the candidates disagree about the tuning value hardly moves, and the share of their disagreements that change the winner collapses.

apiece · Order selection

A model against a window

A block resample's long-run variance is attenuated by its window, and the construction that escapes the window is a model: fit an autoregression and sum its dependence to every lag in closed form. Measured on both halves of the answer, the escape is narrower than it looks. Correctly specified, on 120 rows of a first-order autoregression, the model's error is 45.5% of the truth against the trapezoid block's 42.4% — it removes the bias and buys a spread larger than the bias was — and it pulls ahead only from about 480 rows. Wrongly specified, on a five-period moving average, it is off by 84% to 105% at every length, and even an order chosen by the criterion trails the window throughout.

crossing · Bootstrap

Shocks that are not normal

Every forecast interval in this field reads a normal quantile, and every series it was scored on had normal shocks. Give a first-order autoregression shocks with heavy tails and the 95% interval still covers about 95%; its 99% interval covers 98.2%. Give it skewed shocks and the 95% interval covers 94.8% one step ahead with every miss on one side — 0.0% below, 5.2% above — and the horizon does not repair it: twelve steps ahead the misses are still 0.0% below and 4.9% above, because a persistence of 0.7 keeps the error's skewness at 1.11 however far out the forecast goes. A two-sided coverage count cannot see any of this.

forecast · Forecast

The same dial, on a list that steps by one. The probability that a per-candidate tuning list changes which candidate the table selects, at each of 5 separations, for both tuning parameters at a matched list length of 8, with errors from a five-period moving average. The sieve order runs from 29.5% to 5.8%, a factor of 5.13; the whitening window, from 26.3% to 4.8%, a factor of 5.53. What is held is the number of options, the table, the law, the sample size and the seeds; what cannot be held is the size of a step, since an integer step and a geometric step are different amounts of change. The dial moves both, and it moves them by 1.08 times as much on one as on the other.

The same dial under a second law

On a first-order autoregression, separating the candidates moved the chance that a tuning list changes the winner 2.35 times as far on a whitening window as on a sieve order, and the difference was put down to the step: an integer step is worth a third of a geometric one. Rerun the whole sweep with errors from a five-period moving average and the difference is gone — the probability spans a factor of 5.53 on the window and 5.13 on the order, a ratio of 1.08 — and near the null both lists leave about five and a half candidates within reach, where the autoregression left 3.6 and 1.9. The two factors still point opposite ways under both laws. The step's size was a property of the law the lists were tuned on.

turnover · Order selection

When a longer sample settles the tuning list. How often the tuning list changes which candidate the table selects, against the sample size, for the whitening window (solid) and the sieve order (dashed), on three tables of candidates, twelve hundred draws at each size. Every subset of four predictors: window 12.33% at 120, 2.58% at 240, 1.00% at 480, 0.25% at 960; order 10.58% at 120, 6.25% at 240, 2.33% at 480. The nested sequence of four: window 5.83% at 120, 1.25% at 240, 0.67% at 480, 0.25% at 960; order 5.83% at 120, 4.00% at 240, 2.25% at 480. Six subsets of two predictors each: window 11.75% at 120, 3.75% at 240, 5.75% at 480, 2.33% at 960; order 9.42% at 120, 6.67% at 240, 4.42% at 480. The nested table starts at half the full table's rate and reaches a small one a sample size sooner; the table whose candidates all have the same dimension is still changing its winner on one draw in forty-three at 960 rows.

The table decides when the sample settles it

Letting each candidate regression choose its own whitening changes which candidate wins on about an eighth of draws at 120 rows, and a longer sample settles it — for fifteen nested-and-crossed subsets, at 0.25% by 960 rows. A nested sequence of four candidates gets there a sample size sooner, from half the rate. Six candidates that all hold two predictors never get there: at 960 rows their tuning list still changes the winner on 2.3% of draws, nine times the full table's, because a disagreement among candidates of one size goes on deciding the winner as often at 960 rows as at 240.

apiece · Order selection

What a quarrel between two orders costs. Every two-value list of autoregressive orders from one to twelve, read on the same 2400 draws of a first-order autoregression at a hundred and twenty rows: each cell is the average regret of choosing the order per candidate rather than once for the table, on the draws where the fifteen candidates disagree about it. The cost grows away from the diagonal and stays level along it. Next to the diagonal, a quarrel between neighbouring orders costs 0.00126 for one and two and 0.00088 for eleven and twelve; four orders apart it costs 0.00803 and 0.00802; eleven apart, 0.02436. Every list holding order zero is left off because the candidates never disagree about one.

A quarrel's cost is its width

When fifteen candidate regressions disagree about a whitening's autoregressive order, the disagreement costs in proportion to how far apart the two orders are — 0.0027 of regret for every order of distance, a line that explains 96% of the cost across sixty-six pairs — and not at all by which orders they are: a quarrel between eleven and twelve costs what one between one and two costs, though one of those holds the truth. Fitted on pairs, the line predicts every longer list's cost. The window's quarrels follow no such line.

lists · Order selection

The floor a repair has to choose. Regret of the truncated covariance repaired by flooring its eigenvalues, at a persistence of 0.85 on 120 rows, 120 draws, by floor, on a log scale. Window 4: 1.5649 at 0.001, 0.0945 at 0.01, 0.0385 at 0.05, 0.0285 at 0.1, 0.0226 at 0.2, 0.0235 at 0.4, 0.0292 at 0.7; the taper at the same window 0.0275. Window 8: 1.1919 at 0.001, 0.0980 at 0.01, 0.0334 at 0.05, 0.0258 at 0.1, 0.0197 at 0.2, 0.0209 at 0.4, 0.0249 at 0.7; the taper at the same window 0.0178. Window 12: 0.8692 at 0.001, 0.0621 at 0.01, 0.0239 at 0.05, 0.0209 at 0.1, 0.0212 at 0.2, 0.0239 at 0.4, 0.0282 at 0.7; the taper at the same window 0.0150. Window 20: 1.7110 at 0.001, 0.2495 at 0.01, 0.0399 at 0.05, 0.0238 at 0.1, 0.0202 at 0.2, 0.0219 at 0.4, 0.0259 at 0.7; the taper at the same window 0.0135. Least squares gives up 0.0890 and the whitening told the truth 0.0075.

A covariance repaired

A truncated autocovariance sequence fails to be a covariance matrix on most draws of a persistent series, and this line of essays has always scored it on the draws where it survives. The textbook repair is to floor its eigenvalues. At a floor of 0.001 that moves the matrix by four per cent and gives up 1.56 of regret, seventeen times what least squares gives up, because a whitening reads an inverse and an inverse reads the smallest eigenvalues. At a floor of 0.2 it beats the taper at the shortest windows. So the repair is not a repair: it is a second dial.

banded · Dependence

Which window each reading prefers. The difference between the tapered and the rectangular window, in points, on five readings at four rules: for the two errors a negative number favours the taper, and for the two coverages a positive one does. implied, variance: 1.5, 4.5, -1.1, -2.1. 95% point, uncentred: -6.1, -3.2, -7.1, -6.1. 95% point, centred: 0.9, 3.5, -0.8, -1.0. coverage, uncentred: 5.0, 2.5, 5.8, 4.0. coverage, centred: -1.2, -3.2, 1.0, 0.0. The two uncentred readings favour the taper throughout, and they are the only ones that disagree with the variance.

A shrink towards the answer

This line of essays reported that the 95% point is a dearer reading than a variance, that a further point wants a shorter block, and that a taper beats a rectangle on what a test reads where a variance says the opposite. All three came from one uncentred quantile. Read about the resamples' own mean, the point costs 28.5% against the variance's 45.8%, every level wants about the same block, and the windows order alike on all readings. The taper's coverage had a separate cause: its resampled mean is 0.875 of the sample's, and every series here had a true mean of exactly zero.

readout · Bootstrap

Three charges and two counts. The optimism of an in-sample fit of the widest candidate on 120 rows, in units of its own error variance, over 1500 draws, by persistence. Counted on the 40 rows straight after the sample: 11.1, 21.5, 35.6, 51.5, 65.5, 87.8, 119.1. Counted on rows 600 further on: 9.3, 18.3, 33.0, 50.6, 67.0, 96.1, 159.0. Charged by the long-series inflation: 10.0, 19.3, 34.7, 54.4, 74.3, 114.2, 234.1. Charged by the same inflation summed over the sample's rows: 10.0, 19.2, 34.2, 53.1, 71.8, 108.2, 208.8. Charged by twice the trace of the hat matrix against the errors' correlation on the sample's own design: 10.0, 18.7, 32.0, 47.5, 61.5, 85.2, 132.2.

A charge the sample can afford

The inflation charge for dependent rows runs to twice the counted optimism at persistence 0.95, and the reason offered was that a series of n rows cannot be optimistic about movements slower than n rows. Summing the inflation over n rows does almost nothing: 234 becomes 209, against 119 counted. Evaluating the same variance on the sample's own design gives 132, and moving the fresh rows away from the sample raises the count to 159. So the charge can be made nearly right, and when it is, the regret doubles from 0.032 to 0.061. A selection wants a much larger charge than the optimism, and past about fifty per predictor it is no longer selecting.

proxy · Forecast

Which side the misses fall on. Misses of a nominal 95% forecast interval 1 step ahead on 60 observations of a first-order autoregression at 0.7, split by side. normal shocks: the interval as printed 3.1% below and 3.1% above, shocks from the residuals 3.8% below and 4.2% above, the same, coefficient re-estimated 3.3% below and 4.4% above. Student's t on five: the interval as printed 3.3% below and 3.0% above, shocks from the residuals 3.9% below and 3.0% above, the same, coefficient re-estimated 3.6% below and 3.1% above. skewed shocks: the interval as printed 0.0% below and 6.5% above, shocks from the residuals 4.3% below and 4.1% above, the same, coefficient re-estimated 1.8% below and 4.3% above.

A tail sixty residuals have barely seen

A forecast interval that reads a normal quantile has every one of its misses on one side when the shocks are skewed, and the natural repair is to draw future shocks from the model's own residuals. It works on the side: 0.0% below and 6.5% above becomes 4.3% and 4.1%. It pays for that with coverage, 93.5% to 91.6%, and on sixty observations it does this under normal shocks too. And a nominal 99% interval has a ceiling nothing in the resampling can lift, because one new shock beats all of sixty old ones in either direction with probability 3.3%.

forecast · Forecast

A sequence that knows it was centred

A block length read off a sample's own autocovariances gets worse the further out it reads, because a centred series' autocovariances sum to exactly zero. To first order every lag loses the same amount, about the long-run variance over the sample size, and that can be added back. Corrected, a reading to lag forty goes from 84.5% to 89.2%, and to lag twelve from 93.3% to 94.5%. But the correction overshoots on the long lags, and on the rule that picks its own cut-off, which is short, it moves coverage by three tenths of a point. The identity was the reason the lag mattered. It was not the reason the rule falls short.

taper · Bootstrap

Five 95% intervals for an AR(1) forecast 6 steps ahead at φ = 0.85, fifty observations. Coverage on 4,000 series, the same series for every interval, each simulated interval built from 499 simulated futures. The plug-in formula: 88.48%. The formula, all three repairs: 92.80%. Simulated: parameters drawn: 92.55%. Simulated: residual bootstrap: 88.25%. Simulated: bootstrap, bias removed: 93.47%.

An interval built from simulated futures

A forecast interval read off simulated futures needs no delta method and no normal multiplier, and was expected to close the residual the three repairs to the formula leave. It does not close it on its own: the plain residual bootstrap covers 88.25% six steps ahead at φ = 0.85, the plug-in formula's 88.48% to within the count, because it simulates around the same biased persistence. With that bias removed inside the bootstrap it covers 93.47%, the best of five intervals, and its shortfall stops growing with the horizon — 93.80% at twelve steps, where the repaired formula covers 91.85%.

evaluation · Bias

Named alongside it

The objects these essays reach for when they reach for this one.

Model selection Information criterion Regret Monte Carlo Whitening Bandwidth Selection effect Dependence Coverage Moving average Nuisance parameter Tuning parameter

All concepts