A model against a window

The error no window repairs took apart the error of a block resample’s implied long-run variance. On a hundred and twenty rows of a first-order autoregression at 0.7, the best window’s error was about forty per cent of the truth, and it was made of three parts: the window’s own attenuation of the autocovariances it keeps, the sample’s attenuation of the autocovariances themselves, and the spread from one sample to the next. The spread was the largest part at every size measured, and changing the window moved the total by about four points of forty.

It named the construction that does not have a window at all and declined to measure it. A sieve fits an autoregression to the series and reads the long-run variance off the fit, $\hat\sigma^2/(1 - \sum\hat\phi_j)^2$, so whatever dependence the model contains is summed to every lag exactly, with nothing cut off. The essay’s reason for not putting a number on it was that the comparison is two-sided: a correctly specified model would win, and what a misspecified one costs is a field of its own, so one number would be the flattering half of the answer.

This essay measures both halves on the same footing, and the flattering half turns out not to be as flattering as expected.

Three estimators on the same series #

Two laws, each at unit variance. A first-order autoregression at 0.7, whose long-run variance is $(1 + 0.7)/(1 - 0.7) = 5.67$, and which a first-order sieve describes exactly. And a five-period moving average — each value the average of five consecutive shocks — whose long-run variance is 5 and which no finite autoregression describes exactly.

Three estimators on every series:

• a first-order sieve — Yule–Walker at order one, correctly specified under the first law and wrong under the second;
• a sieve with its order chosen by Akaike’s criterion from zero to twelve, which is the sieve as it is used, never exactly right under the second law and free to overfit under the first;
• the trapezoid block resample at whichever block length gives it the smallest error on that law and size — the best any window in this field does, handed that advantage on purpose, because the comparison is meant to be against the window at its best.

Every error is a share of the truth: the bias, the spread across six hundred draws, and their root sum of squares.

Under the law the model describes #

The first-order sieve does what a correctly specified model should do to the bias. It is −5.5%, against the trapezoid block’s −33.0% at its best length of twelve rows: the model sums the geometric dependence to every lag, and what is left of its bias is the downward bias of a lag-one autocorrelation estimated from a hundred and twenty rows.

And its spread is 45.2%, against the block’s 26.5%. The total error is 45.5% against 42.4%. On a hundred and twenty rows the correctly specified model is the worse estimator.

The reason is the shape of the formula. The sieve’s long-run variance at order one is $\hat\gamma_0 (1 + \hat\rho)/(1 - \hat\rho)$, and its sensitivity to the persistence is steep: at 0.7 the factor’s derivative is $2/(1 - \rho)^2$, about 22, so a sampling error of 0.07 in $\hat\rho$ — one standard error at this length — moves the estimate by more than a quarter. The block resample never divides by $1 - \hat\rho$. It sums a window’s worth of sample autocovariances directly, which is biased because it stops, and stable because nothing in it is amplified.

The sieve whose order the criterion chooses is worse again: 50.0% at a hundred and twenty rows. Its median order is one, the right one, but it averages 1.66, so on many draws it chooses more, and every extra coefficient adds its own noise to $1 - \sum\hat\phi_j$.

As the series grows #

At 240 rows the first-order sieve and the block are level: 33.3% and 32.9%. At 480, 23.1% against 25.4%; at 960, 15.8% against 19.5%. The model’s spread falls like one over the square root of the length, as every spread does; the block’s error falls more slowly, because its best block length grows with the sample — twelve rows at 120, twenty-four at 960 — and it is always trading some bias against its spread. Its bias is still −12.2% at 960 rows, where the model’s is −0.4%.

So the correctly specified model does escape the window’s ceiling, and the escape begins at about two hundred and forty rows of this law. Below that the window wins, because the thing the model removes — the bias — is smaller than the thing it adds. The criterion-chosen sieve is behind the first-order one at every length, by 3 to 6 points, and catches the block only at 960 rows.

Under the law no model describes #

The other half is not close. On a hundred and twenty rows of the moving average the first-order sieve’s bias is +68.3%, its spread 79.7% and its error 104.9%: it reads a lag-one correlation of about 0.8, the moving average’s own, and sums it as though it decayed geometrically, which this dependence does not — it stops dead after the fourth lag. The block’s error is 35.8%. The criterion-chosen sieve removes the bias almost entirely, −0.7%, by choosing long autoregressions, and pays for them in a spread of 78.2%.

And it does not improve the way the other law did. The first-order sieve’s error is 94.8%, 87.3% and 83.6% at 240, 480 and 960 rows: its spread falls and its bias does not, settling near 79%, which is what a geometric decay at 0.8 overstates a moving average’s long-run variance by. The criterion’s sieve falls to 63.5%, 44.2% and 33.7%, and its bias grows from 9.3% to 13.9% as its chosen order reaches the top of the range. The block falls to 26.7%, 20.8% and 15.3%. At every length the window is better than either model, and at 960 rows it is better than the criterion’s sieve by a factor of two.

Why this moving average defeats every autoregression #

The moving average here is not an arbitrary choice and it is worth seeing why it is so hard. Each value is $(u_t + u_{t-1} + u_{t-2} + u_{t-3} + u_{t-4})/\sqrt 5$, so its moving-average polynomial is $1 + z + z^2 + z^3 + z^4 = (1 - z^5)/(1 - z)$, whose roots are the four fifth roots of unity other than one. They lie on the unit circle. An autoregression approximates a moving average by inverting its polynomial, and a polynomial with roots on the unit circle has no convergent inverse: the autoregressive representation’s coefficients do not decay. So every finite autoregression is wrong by a margin that falls only slowly with its order, which is why the criterion keeps asking for more — a median of eleven lags at a hundred and twenty rows, and twelve, the ceiling, at 960 — and why the bias it removes comes back as the cap binds.

The window has no such difficulty, because a dependence that stops at the fourth lag is exactly what a window is good at: once the block is long enough to hold four lags, every autocovariance the law has is inside it, and the only bias left is the window’s own taper and the sample’s attenuation.

That is the asymmetry the earlier essay was guarding against, measured. A model is a statement about the shape of the dependence, and the window is a statement about its reach. Where the shape is right the model wins eventually; where the reach is short the window wins at once; and a law can be short in reach and wrong in shape at the same time.

The window’s own trade, block by block #

The block estimate quoted so far is the window at its best length, and the comparison is fairer for seeing what that best is a minimum of. On a hundred and twenty rows of the autoregression the trapezoid’s bias at blocks of four rows is −60.4% and its spread 11.4%; at twelve, −33.0% and 26.5%; at twenty-four, −24.8% and 39.4%; at forty-eight, −30.1% and 50.4%. The bias shrinks until the blocks are a fifth of the series and then grows again, because long blocks are few and the sample’s own attenuation of its autocovariances, which every window inherits, is worst at the long lags those blocks reach. The spread grows throughout. The error is flat across a wide range — 42.4% at twelve rows, 42.5% at sixteen, 44.3% at twenty — which is why the window’s best length is not a precise choice and does not need to be.

The first-order sieve sits on that same trade at one end: its bias is the smallest any estimator here has, and its spread is larger than the window’s at any block length up to thirty-two rows. In that sense the model is not a different kind of estimator from the window. It is the window taken to an infinite block, with the sample’s autocovariances replaced by a fitted geometric sequence that does not attenuate — the end of the trade the window cannot reach, reached by assumption.

Under the moving average the window’s minimum is lower and sits at the same place: −19.8% and 29.8% at twelve rows, an error of 35.8%, with 36.5% at eight and 39.0% at sixteen. A dependence that stops at the fourth lag is held almost entirely by a block of eight, and what is left of the bias there is the window’s taper and the sample’s attenuation, not dependence the block failed to reach.

What the model is buying, and what it costs #

Put the two laws together and the comparison has a simple form. The model removes the window’s bias, which at a hundred and twenty rows is a third of the truth under the autoregression and a fifth under the moving average. In exchange it multiplies the spread, because every quantity it computes passes through $1/(1 - \sum\hat\phi_j)^2$, and a persistence estimated from a short series is noisy exactly where that factor is steep.

So the model’s case depends on how large the bias is that it removes against how steep the amplification is that it adds. At 0.7 on a hundred and twenty rows they are nearly equal and the model loses by three points. At higher persistence the window’s bias grows — blocks short enough to be stable reach a smaller share of the dependence — but so does the amplification; at lower persistence both shrink. The triangle that was not the multiplier found every block construction attenuated by the same triangle at the first lag; the sieve is the one construction that is not attenuated at all, and that is precisely the property that makes it unstable.

The length nobody has found the best block length for a variance estimate wandering by sixteen rows from draw to draw. The sieve has a dial too, its order, and under the moving average it is chosen at the ceiling. A practitioner who reads either estimator’s output without its dial has read half of it.

What a practitioner should use #

The measurement points in one direction more strongly than the earlier essay could have said.

Measure both halves before choosing. The two laws here have the same lag-one correlation to within a tenth and long-run variances within an eighth of each other; a practitioner looking at a correlogram’s first bar could not tell them apart, and the right estimator differs between them by a factor of three in error. The observations that repeat each other made the general case that a lag-one correlation is a poor summary of a dependence; here it is the whole of what decides whether a model helps.

On a short series, the window. At a hundred and twenty rows the trapezoid block is the better estimator under both laws, including the one the model describes exactly. A sieve is not a safer choice on a short series; it is a riskier one, whose variance is amplified by exactly the persistence that makes the long-run variance matter.

On a long series of a law with a geometric shape, the model. From about two hundred and forty rows of a first-order autoregression the correctly specified sieve is level with the window and pulls ahead, by four points of error at 960 rows, with almost no bias.

And never a model chosen to be general. The criterion-chosen sieve, which a practitioner reaches for precisely because the law is not known, is the worst of the three under the first law at every length and behind the window by a factor of two under the second. Generality bought by choosing an order is not generality; it is a second selection problem whose cost the order the tail is drawn at measured for a different purpose and found to be real.

Six hundred series shared by the three estimators #

Six hundred draws at each law and length, on one seed shared by the three estimators so every comparison is on the same series. The sieves are Yule–Walker fits; the long-run variance at order $p$ is the innovation variance over $(1 - \sum\hat\phi_j)^2$, with the denominator floored at $10^{-6}$ for the rare draw whose coefficients sum to one. The criterion is $n\log\hat\sigma_p^2 + 2p$ over orders zero to twelve. The block estimate is the trapezoid window’s implied long-run variance, $\sum_{|k|<\ell}\kappa(k)\hat\gamma(k)$, at each of nine block lengths from four to forty-eight, and the one reported is the length with the smallest error on that law and size: twelve, sixteen, twenty and twenty-four rows under the autoregression, twelve, twelve, sixteen and sixteen under the moving average. Nothing is resampled; the implied variance is exact given the series, which is the identity the field’s first essay is built on.

A sieve that wins on its own law from 480 rows, and nowhere on a moving average #

The claims are that on 120 rows of a first-order autoregression at 0.7 a correctly specified first-order sieve estimates the long-run variance with an error of 45.5% of the truth against the trapezoid block’s 42.4% at its best length, because its spread is 45.2% against 26.5%; that it draws level at 240 rows and leads from 480; that under a five-period moving average the first-order sieve’s error never falls below 83% and its bias settles near 79%; that a sieve whose order is chosen by Akaike’s criterion is behind the window at every length under the moving average and behind the first-order sieve under the autoregression; and that the moving average defeats every finite autoregression because its moving-average polynomial has its roots on the unit circle.

Not claimed: anything about a block weighted inside itself at other windows, which would move the block’s column by the few points the earlier essay found windows worth; anything about a sieve bootstrap’s critical values or coverage, which are what a sieve is usually used for and which depend on more than the long-run variance; anything about persistences other than 0.7; and anything about long memory, where neither construction has a finite target on a series this short.

Still open: the persistence at which the model starts to pay #

At 0.7 the correctly specified model and the window are level at about two hundred and forty rows. The model’s amplification grows like $1/(1 - \rho)^2$ and the window’s bias, at its best length, grows with the reach of the dependence, so the crossing should move with the persistence — later at 0.9, where the amplification is twenty-five times its value at 0.5, and earlier at 0.5, where a block of a few rows already holds nearly all the dependence. The measurement that would turn this into a rule is that crossing as a function of the persistence and the length together, with the persistence read from the sample as a practitioner would read it, since the length at which a model is worth fitting is itself a function of the number the model is fitted to estimate.