What the model says next

Three fields on this site are about series in time and all three stop in the same place. The dependence field measures what correlation between neighbours does to a standard error. The cointegration field estimates a long-run relation between two series. The systems field counts how many such relations three series carry. Every one of them ends with an estimate and a statement about how uncertain it is.

None of them forecasts. And forecasting is not the same question asked again — it is a different question, because the thing being guessed at is not a parameter that exists somewhere waiting to be pinned down. It is an observation that has not happened, and it carries its own noise on top of whatever uncertainty the estimate has.

Two things are being added, and only one of them is estimation error #

Write the model down first, because everything below is arithmetic on it. A first-order autoregression says each observation is a fraction of the last one plus a fresh shock:

xₜ = φ·xₜ₋₁ + εₜ

with the shocks independent and of variance σ². Everything about the series follows from those two numbers.

Now ask what the model says about xₙ₊₁, standing at time n with xₙ in hand. The shock εₙ₊₁ has not happened yet and nothing in the data says anything about it, so the best available statement is its mean, which is zero. That leaves

x̂ₙ₊₁ = φ·xₙ

and an error of exactly εₙ₊₁, whose variance is σ². One step ahead, the forecast error is one shock and nothing else.

Two steps ahead the same reasoning applies twice. xₙ₊₂ = φ·xₙ₊₁ + εₙ₊₂ = φ²·xₙ + φ·εₙ₊₁ + εₙ₊₂, so the forecast is φ²·xₙ and the error is a weighted sum of the two shocks that have not happened yet, with weights φ and 1. Its variance is σ²(1 + φ²).

Carry that out to h steps and the pattern is a sum:

x̂ₙ₊ₕ = φʰ·xₙ, and Var(error) = σ²·(1 + φ² + φ⁴ + ⋯ + φ^(2(h−1)))

The weights are the model’s own recursion applied to a single unit shock, and they have a name — the ψ weights — that matters because they are the same object for any autoregression of any order. ψ₀ = 1, and ψⱼ = Σᵢ φᵢ·ψⱼ₋ᵢ. The forecast error h steps ahead is Σ ψⱼ·εₙ₊ₕ₋ⱼ, a sum of h independent shocks with those weights, and its variance is σ²Σψⱼ².

At φ = 0.7 and σ = 1 that sum runs 1.0000, 1.4900, 1.7301, 1.9054 at one, two, three and five steps.

The band stops widening, and where it stops is not a coincidence #

Those four numbers are converging on something. At ten steps the variance is 1.9592 and it is still climbing, but not by much; the limit is

σ²/(1 − φ²) = 1.9608

which is the unconditional variance of the series — how far it wanders from its own mean over the long run, computed without reference to any forecast at all.

That is not an accident of the arithmetic and it is worth stopping on, because it is the one structural fact about forecasting a stationary series. A forecast is a statement conditional on what has been seen. As the horizon grows, the conditioning stops carrying information: φʰ goes to zero, the point forecast goes to the mean, and the interval opens out to whatever interval a person who had seen nothing at all would have quoted. Past a certain horizon a model with a fitted parameter and a person who knows only the marginal distribution give the same answer, and the model has nothing further to lose.

Which horizon is a question with an answer, and it is the same φ doing the work. At φ = 0.3 the series has forgotten where it was after three or four steps and the fan is a rectangle almost immediately. At φ = 0.97 it is still remembering at fourteen.

This is the shape of every forecast interval that anybody publishes, and it is the reason those intervals look reassuringly bounded on a chart. The boundedness is not the model being confident. It is the model running out of things to say.

It is not a confidence interval, and the difference is most of its width #

Everything this site has counted so far has been an interval for a parameter: a proportion, a slope, a population spread, the location of an optimum. Those intervals shrink like 1/√n, and the whole of the intervals field is about whether they contain what they claim to contain at the rate they claim.

A forecast interval is not one of those and does not behave like one. It contains a shock that has not happened, and no amount of data makes a shock smaller. Set n to a million and the interval above is still ±1.96σ at one step, because σ is a property of the world rather than of the sample. The part of it that shrinks with n is the estimation error, and at any sample size worth having that part is the smaller half by a wide margin.

That has a consequence worth stating plainly, because it is the most common misreading of a chart with a fan on it: collecting more history does not narrow a forecast interval to a point. It narrows it to the band the model’s own noise dictates, and every additional observation after that buys a sharper estimate of a width that is not going to change. A reader who has internalised 1/√n from every other field on this site will over-read what a longer series buys here, and the figure above is the corrective — the two lines converge and neither goes to zero.

The refusal that makes the previous paragraph checkable #

Two claims have been made about the ψ sum and neither has been tested: that σ²Σψ² is the right variance, and that it never exceeds the unconditional variance. Both are checked in the field’s own library, and the way they are checked is the site’s usual arrangement rather than an inspection of the algebra.

The first is a simulation with no fitting anywhere in it. Generate the series, forecast it with the true φ, and take the mean squared error over six thousand series at horizons one, three and eight. The counted numbers agree with the closed form to within four per cent, which is the simulation’s own precision, and the two routes share no arithmetic: one runs a recursion over weights and the other runs a random number generator.

Doing it with the true parameters is deliberate and it is the point of the next essay. With φ known there is nothing to be wrong about and this is a check on the formula. With φ estimated the same formula is a different claim entirely, and it is false.

The forecast is one number and the forecasts are a distribution #

Everything so far concerns one series. The picture at the top of this page is one draw, and a reader looking at it has been given no way to tell which of its features are the model and which are the particular sixty numbers it was fitted to.

So here are twenty. Same model, same φ, same σ, twenty different series, each fitted separately and each forecast forward.

Two things are visible there that the first figure cannot show. The paths start from different places, which is expected — they are different series. And they decay at different rates, which is not a property of the model at all: the model has one φ. It is a property of the estimation, and it is the subject the next essay takes up, because the spread of those decay rates is exactly the uncertainty the band in the first figure does not contain.

There is also something systematic hiding in that spread, and it is easier to state than to see: the paths return to the mean faster than the series they came from. Least squares estimates the persistence of a series as smaller than it is, the h-step forecast raises that estimate to the power h, and a proportional error raised to a power is a larger proportional error. That is a bias in the point forecast rather than a shortfall in the interval, and it is a second defect in the same arithmetic.

How much of the band is left, at every horizon #

The four variances converging on 1.9608 are a geometric series, so the distance still to be travelled has a closed form and it is the tidiest expression in this essay.

The variance at horizon h is $\sigma^2(1 - \varphi^{2h})/(1 - \varphi^2)$, and the limit is $\sigma^2/(1 - \varphi^2)$, so the share of its final variance the band has reached is exactly $1 - \varphi^{2h}$. At φ = 0.7 that is 0.510, 0.760, 0.882, 0.972 and 0.99918 at one, two, three, five and ten steps — which reproduces 1.0000, 1.4900, 1.7301, 1.9054 and 1.9592 to the last digit each.

Read as a rate: what the band has left to grow halves every 0.347/(−log φ) steps — about once per step at φ = 0.7, once every 2.1 steps at φ = 0.85, and once every 11.4 steps at φ = 0.97. That single number is the whole shape of the fan, and it is a property of φ rather than of the sample, the horizon or anything anybody chooses.

It also converts the informal reading of the figures into arithmetic. Reaching 95% of the final variance takes 1.3 steps at φ = 0.3, 2.9 at 0.6, 4.2 at 0.7 and 49 at 0.97 — so the essay’s “three or four steps” and “still remembering at fourteen” are the same expression at two ends of its range. A reader reading the ceiling off a picture is applying a stricter threshold than 95%, which is why the figures look like eight steps where the formula says four; the criterion is theirs and the rate is not.

Both edges move as 1/n, and neither arrives #

The winning band runs from 0.1186 to 0.9231 at fifty observations, so a fitted model is the right choice on 80% of the stationary range. The upper edge’s leading form says how that changes.

1 − (n − k)/(n + k) = 2k/(n + k), which at k = 2 is 4/(n + 2). So the gap between the upper edge and one closes as 4/n: it is 0.077 at fifty observations, 0.033 at a hundred and twenty, and 0.004 at a thousand. Pushing the upper edge to 0.99 takes about four hundred observations, and pushing it to 0.999 takes four thousand.

The lower edge moves the same way and from the other side, so at a hundred and twenty observations the band covers about 89% of the range against fifty’s 80%. Both edges are still moving at every length, and the reason neither arrives is structural rather than numerical: at φ exactly one the series is a random walk and carrying the last value forward is not merely competitive but optimal, and at φ exactly zero the observations are independent and the mean is optimal. The two benchmarks are each exactly right at one point of the range, so a fitted model that estimates two numbers can only approach them there and never beat them.

That is worth carrying because it says what the band is a picture of. It is not a region where the model is good; it is the region where the cost of estimating two parameters is smaller than the distance from the nearer of two exactly-right answers.

Before any of this: is a model worth fitting at all #

There is a question logically prior to everything above, and it is usually skipped because the answer seems obvious. A forecast has to be compared against something, and the two things it has to beat are both embarrassingly cheap.

The last value carried forward. Guess that tomorrow is the same as today. This estimates nothing, has no parameters and cannot be misspecified. Its mean squared error is E[(xₙ₊₁ − xₙ)²] = 2γ₀(1 − φ), where γ₀ is the unconditional variance — an exact expression with no estimation in it anywhere.

The mean. Guess the long-run average and ignore the data’s order entirely. Its error is the unconditional variance γ₀, plus what it costs to estimate the mean of a dependent series.

The fitted model has to beat both, and it pays for what it uses: a first-order autoregression estimates two numbers, and estimating them inflates its one-step error by roughly σ²(1 + 2/n).

At fifty observations the band runs from φ = 0.1186 to φ = 0.9231. Below the first, a series so close to noise that fitting a dynamic parameter costs more than the dynamics are worth, and the sample mean is a better forecast. Above the second, a series so persistent that yesterday’s value is a better forecast than the fit, because the fit spends variance estimating a coefficient that is nearly one and the naive rule assumes it is one and pays nothing.

The upper edge has a leading-order form worth writing down because it is a function of n alone: φ = (n − k)/(n + k) with k the number of parameters estimated, which is 0.9231 at n = 50 and k = 2, and matches the solved root to four figures. The lower edge’s leading form is √(k/(n + k)), which comes out at 0.1961 and is an over-estimate — it drops what estimating the mean of a dependent series costs, and putting that back moves the crossing to 0.1186. The approximation is named as one rather than quoted, which is the difference between a formula and a measurement.

Both edges move outward as n grows and neither ever reaches the end of its range. There is no sample size at which fitting a model is unconditionally the right thing to do, and where the exceptions are is computable before any data exists.

What is being claimed here, and what is not #

This field takes forecasting from a fitted linear time-series model, and the selection of its order: the point forecast, the interval, what estimation costs both of them, and the two benchmarks. That was named as not claimed when the cointegration and systems fields were written, and it is claimed now.

What stays out, and is named so that a later phase knows it was a decision: seasonal models, state-space and Kalman filtering, exponential smoothing as a method in its own right, non-linear and volatility models, and forecast evaluation as a formal testing problem — comparing two forecasters’ accuracy is a hypothesis test with its own distribution theory and none of it is here. Forecasting a cointegrated system, where the long-run relation is what carries the horizon, is measured in the pair’s own field and is not repeated.

The boundary against the dependence field is the one that needed drawing. That field is about what correlation does to an estimate of a mean; this one is about what it does to a statement about a future observation. They share the machinery and ask opposite questions of it: there, dependence is the thing that spoils an interval, and here it is the only reason a forecast is possible. The same φ that makes fifty observations worth about six is what makes a forecast at horizon three better than a guess, and neither field is complete without the other’s sign on it.

Two neighbouring results are worth naming so that a reader does not go looking for them here. That a series which is not stationary — a random walk — has a forecast error variance growing without any limit at all is the unit-root case, and what it does to a regression between two such series is the spurious one rather than a forecasting question. And what differencing costs when it is applied to a series that did not need it is measured in the dependence field, where the answer is a variance ratio rather than a forecast.

The checks, and the refusal that makes them mean something #

Three claims are gated in this field’s library and they are the three the essay makes. The ψ sum matches its simulation at three horizons. The variance grows with the horizon and stays below the unconditional variance, which is checked as an inequality rather than assumed from the algebra. And this file’s autoregression is required to agree with the one the dependence field has had from the start — same marginal variance, same lag-one autocorrelation — because two functions with the same content under different names is the divergence worth preventing, and the two are not identical here: one takes a scalar φ and one a vector, and the check is that they agree where they overlap.

The refusal is the interval a printout invites. A fitted model reports one residual standard deviation, and it is the only spread on the output; using it at every horizon is right at h = 1 and wrong everywhere else. At six steps on a persistent series that interval covers 75.9% where it claims 95%, against 92.5% for the same forecast given the horizon’s own variance. The check requires the first to fail, because a check that has never rejected anything proves nothing.