Normal Distribution
nothing is called 95% until it has been counted
Concept

Stationarity — where it appears

That a series' distribution does not change over time, which almost every method for dependent data assumes. Testing for it and testing for its absence are different questions with different defaults, and failing to reject is not the same as establishing it.

Named by 23 essays across 9 fields — each of them below, with the objects they name alongside it.

Twenty series with a lag-one correlation of 0.8. Every series has a true mean of zero and 60 observations. The marks on the right are the twenty sample means. The variance of that mean is 8.3 times what 60 independent observations would give, so the series is worth about 7 of them.

The observations that repeat each other

Almost every standard error divides by √n, which claims the observations carry independent information. At a lag-one correlation of 0.8 a fifty-point series is worth about six independent observations, and its 95% interval covers 47%.

timeseries · Dependence A pair pulled back at 20% of the gap per step. Above, the two series. Below, the difference between them. The gap is pulled back towards zero by 20% of itself each step, so it stays inside a band of 14.3 while the series themselves travel much further. Nothing here is stationary except the difference. The faint line below is the gap for two free walks from the same seed, drawn for comparison.

The regression that is not spurious

Two random walks regressed on each other are called significantly related three times in four, so the time-series field ends in a warning. The exception it names and does not measure is here — and when the pair is genuinely tied, the fitted relation converges at rate 1/n rather than the usual 1/√n.

cointegration · Spurious Three series and one relation between them. Above, three series generated from Δy = Πy₋₁ + ε with Π of rank 1. Below, the combination y1 −y2. It stays inside a band of 9.5 while the series themselves travel 28.4. The count of combinations that behave this way is the rank of Π, and it is what every method in the field sets out to estimate.

Three series and a count

A pair of series is either tied together or it is not, so its whole inference is one test with one answer. Three can carry none, one or two relations at once — and the thing being estimated stops being a slope and becomes an integer, read off the gap in a spectrum whose top eigenvalue holds at 0.25 while the rest fall like 1/n.

systems · Rank One forecast, and the band the arithmetic puts round it. An AR(1) with φ = 0.75, 60 observations, fitted by least squares and forecast 14 steps ahead. The point forecast decays towards the fitted mean at φ̂^h; the band is ±1.96 standard errors from σ̂²Σψ̂², which grows with the horizon and stops at the unconditional spread 1.72. The dashed pair is the same band computed at the true parameters, which nobody has. The marks past zero are what actually arrived: 12 of 14 inside the band this once, which is one draw and settles nothing.

What the model says next

The usual account of a time series stops at estimation. A forecast asks the other question — not what the parameter is but what the next observation will be — and the band round it is a closed form that grows with the horizon and then stops growing, at a value the series was going to reach anyway.

forecast · Forecast The distribution of the largest statistic in the table. Fit the benchmark to the whole series, resample its residuals, simulate 199 series in which the null is true by construction, re-run the entire eight-variant search on each, and keep the largest statistic. That is the distribution drawn here, and it is the distribution of the thing a specification search actually reports. It is centred at 1.045 — the maximum of eight statistics is not centred at zero however well each of them behaves — and its 5% point is 2.536. A table read against 1.671 is reading the distribution of one statistic; a Bonferroni correction reads it against 2.577 and is nearly right here, because eight variants that each add a different lag are nearly eight separate chances.

A null with a model in it

The distribution to read the winner of a table against cannot be resampled from the data, because the data does not contain the null. It has to be generated from a model — which is the assumption the resampling was chosen to avoid.

search · Bootstrap What a 95% forecast interval covers, counted. 1200 series of 25 observations from an AR(1) with φ = 0.7, at each horizon, on one set of seeds. The upper line is the interval computed at the true parameters — it covers 95.3% on average, which is the check that σ²Σψ² is the right formula rather than a claim about anything a forecaster can do. The lower line is the same formula fed σ̂² and φ̂: 92.8% at one step and 87.3% at 6. The interval that would cover what it claims is 6.9% wider at one step.

The interval that forgets it estimated

The forecast band is derived for a model whose parameters are known, and then computed by putting estimates into it. Counted, the 95% interval covers 87.3% six steps ahead on twenty-five observations, and the point forecast inside it returns to the mean a third faster than the series does.

forecast · Forecast Two independent random walks, 100 steps. Nothing connects these two series: each is generated from its own independent draws. Regressing one on the other gives a slope with t = -10.9, R² = 0.55 and p = 0.0e+0 — a result that would be reported as a finding by any standard output.

Two walks and a finding

Regress one random walk on another, independently generated, and the slope is significant 76.7% of the time with a median R² of 0.17. Nothing connects the two series, nothing in the output says so, and more data makes it worse.

timeseries · Spurious Generality in the wrong direction buys nothing. Regret on a sample whose persistence changes from 0.95 to 0.65 at row 60, over 200 draws. The three stationary rules — told one number, told a window, told an order — are within 0.4 standard errors of each other, and all three stop in the same place: they are general in the lag direction, and the departure is in the other one. Letting the model change once, at a point estimated from the same residuals, is worth 0.05021 more at 4.5 paired standard errors — about as much again as the whole of the first repair. Being told where the break is adds 0.01926, and being told the entire covariance adds 0.02465.

Where the generality runs out

A covariance that changes half way through a sample is not one a window can estimate. One number, a window and an order are worth the same as each other on it — and letting the model change once, at a point nobody can locate, is worth as much again as all three.

general · Dependence The crossing is in the dependence, not in the split. Regret of each rule as the design and the errors are made persistent at the same coefficient, scored on fresh rows because the closed form assumes exactly what is being taken away. An optimism theorem counts rows; when the rows repeat each other there are fewer of them than there are rows, the penalty is too small for the fit it is correcting, and the criterion starts buying coefficients it should not — its average winner grows from 3.31 coefficients to 3.90. The hold-out never used the theorem and overtakes at ρ ≈ 0.81. Schwarz's criterion, worst of the three on independent rows, is best on repeating ones — its heavier penalty is right for the wrong reason.

Where the two searches cross

The obvious dial between a criterion and a hold-out is how much of the sample to hold out, and moving it never changes the answer. The dial that does is one nobody chooses — how much each row repeats the one before it — and the two rules change places at about 0.81.

proxy · Forecast The same data, one regression per choice of left-hand side. The two-step procedure has to put one series on the left, and with 3 series there are 3 ways to do it. Each returns a relation and a residual test; the 5% point is -3.71, simulated. Here they do not agree: 2 of 3 reject, and the relations they report are written with a 1 in the position of whichever series was on the left, so they can be compared. Nothing in a printed output records which regression was run.

Which series goes on the left

The two-step procedure has to pick a series to regress the others on, and nothing in its output records which. With a pair that choice never changes the verdict. With three series and one relation between them, the three choices disagree about whether the system is cointegrated at all 98.0% of the time.

systems · Rank Least squares estimates persistence low, by an amount with a formula. 3000 series of 50 observations at each persistence. The lower curve is the counted bias of the least-squares estimate of φ, and the open marks on it are −(1 + 3φ)/n, computed rather than fitted. The upper curve is the bias left after adding that quantity back, evaluated at the estimate rather than at the truth nobody has: -0.0020 at φ = 0.3, -0.0023 at φ = 0.5, -0.0039 at φ = 0.7, -0.0059 at φ = 0.8, -0.0108 at φ = 0.9, -0.0165 at φ = 0.95. The formula is a leading-order expression and it understates the bias where the persistence is nearest one — -0.0882 counted against -0.0770 predicted at φ = 0.95, which is the corner of the parameter space every one of these approximations is worst in.

Correcting the persistence

Least squares estimates how much a series remembers of itself as smaller than it is, at every value it can take, by an amount with a closed form. Subtracting that amount back is one line of arithmetic, and what the line costs is variance.

evaluation · Bias The trace statistic under the null, and the 5% point it needs. 600 systems of 3 unrelated random walks, each put through the reduced-rank regression, with the statistic for "rank ≤ 0" collected. The 5% point is 31.91. There is no standard table to look that up in: the distribution depends on the number of common trends under the null and is not a chi-square, so the value is simulated on one set of seeds and applied on another — exactly the position the pair's residual test was in one field ago.

Counting what is still wandering

The statistic that turns a spectrum into an integer has one name and three distributions. Its 5% point is 8.12, 18.64 or 31.74 depending only on how many series are left wandering under the null being tested — and read against the wrong one of those three, it calls unrelated random walks cointegrated most of the time.

systems · Rank The correction, at a generating α of -0.2. Each point is one step: the gap at the end of yesterday against the change in y today. The fitted slope is -0.202 against the -0.2 the data was generated from, which means 20% of any disagreement between y and its long-run relation with x is undone in a single step. A shock therefore has a half-life of 3.1 steps. Neither series is stationary; the relation between them is.

The model that corrects its error

A cointegrated pair can always be written as a mechanism — today's change in y depends on yesterday's disagreement between y and its long-run relation with x. The coefficient of that disagreement is recovered from data that never saw it — and on unrelated series the same fit produces one a t table would call real 41% of the time.

cointegration · Dependence What differencing fixes, and what it costs, 100 steps. The first pair is the false-positive rate for two independent random walks: 77% on the levels, 4.9% on the differences. The second pair is how much of a real relationship survives: R² falls from 0.91 to 0.33. The same operation does both.

What differencing costs

Differencing takes the false-positive rate between two unrelated walks from 76.7% to 4.9%, and takes a genuine relationship's R² from 0.91 to 0.33. Applied to a series that did not need it, it doubles the variance and installs a correlation of −0.5 that the data never had.

timeseries · Spurious What the long-run relation is worth, at α = -0.2. Root mean squared one-step forecast error of the error-correction model divided by that of the model fitted on differences alone; below one means the levels helped. With the equilibrium known the ratio is 0.929 at 100 observations and settles on 0.905 by 3,200, against a closed form of 0.905 that mentions no sample size at all; the excess at short series is the cost of fitting three coefficients on fifty observations. With the equilibrium estimated as well it is 1.127 at 100 — worse than differencing — and 0.914 at 3,200. The gap between the two curves is the cost of not knowing β.

The cost of differencing a pair

Differencing two cointegrated series makes every standard error honest and throws away the one thing known about where they are going. The error-correction model forecasts better by exactly what a closed form says — and at four hundred observations it is better on four series in five and worse on average.

cointegration · Dependence The correction does not arrive at the truth, it passes it. The average decay factor a forecast applies to the last observation, at φ = 0.85 and 50 observations, 3000 series per horizon. The middle curve is φʰ, what the model actually does. Below it is the uncorrected forecast, which uses φ̂ʰ and reverts too fast — 24.8% short at h = 4, 30.0% short at h = 6, 32.7% short at h = 8. Above it is the forecast built on the corrected estimate, which overshoots, and the reason is arithmetic rather than a bad correction: raising an unbiased estimate to a power does not give an unbiased estimate of the power, and the higher the power the more the spread of φ̂ is converted into overshoot.

The repair that moves the wrong number

Correcting the bias in a persistence parameter is one line of arithmetic that works. Feeding the corrected estimate into a forecast repairs the number everybody looks at, makes the forecast worse by squared error at moderate persistence, and improves the interval for a reason that has nothing to do with bias.

evaluation · Bias Every equation's adjustment speed, and the one number they make together. Each series gets its own equation, each is regressed on the same lagged disequilibrium, and what comes back is the whole vector α. Averaged over 400 systems at n = 300: α₁ = -0.154 against -0.15 generated, α₂ = 0.104 against 0.1 generated. The gap closes at the combination of them rather than at any one entry — 25% of any disagreement per step, a half-life of 2.41 steps, where the single equation that fits only the first series reports 4.27.

Which series does the moving

“y adjusts towards x” and “x adjusts towards y” are different mechanisms with identical long-run relations, and a single-equation model cannot tell them apart because it only writes one equation. Writing all of them recovers a vector — and a gap that closes at 25% a step where one equation alone reports 15%.

systems · Adjustment The average decay factor each route produces, φ = 0.85, 6 steps ahead. The truth is φ^6 = 0.3771. no correction averages 0.2616 with a spread of 0.1646 and a squared forecast error of 3.2516; the formula, on the persistence averages 0.4213 with a spread of 0.2528 and a squared forecast error of 3.4827; the bootstrap, on the persistence averages 0.4355 with a spread of 0.2655 and a squared forecast error of 3.5120; the bootstrap, on the decay factor averages 0.3375 with a spread of 0.2278 and a squared forecast error of 3.4132. 800 series, 100 bootstrap refits each.

Correcting the forecast instead

The complaint against the usual repair is that a correction aimed at the persistence lands on the wrong quantity. Aiming it at the decay factor the forecast actually uses fixes exactly that — the error stops compounding with the horizon, 69.7% becomes 9.5% at twelve steps — and the forecast still gets worse.

evaluation · Bias The damage and the warning, against the same dial. Two readings at each persistence. In the darker colour, how often a regression between two independent series of 200 steps is called significant at 5%: 4.9% at φ = 0, 34.2% at 0.8, 52.4% at 0.9, 83.4% at a unit root. In the lighter, how often the standard unit-root test refuses a unit root on one of those series — the chance the analyst is told the series is stationary and may be regressed: 87.2% at φ = 0.9 and 31.9% at 0.95. At φ = 0.9 both are high at once, which is a correct diagnostic licensing a regression that is wrong half the time.

The cliff that is a slope

A regression between two independent series is called significant 4.9% of the time at no persistence, 52.4% at a lag-one correlation of 0.9, and 83.4% at a unit root. The rule the field offers asks whether the last of those holds, and at 0.9 the unit-root test correctly refuses one 87.2% of the time.

timeseries · Spurious The exceedances arrive together. 300 steps of a max-autoregression with dependence 0.75, drawn on a logarithmic scale because its marginal has no variance. The rule marks the 0.9 quantile: 30 of the 300 readings are above it and they fall into 5 clusters, the largest holding 11. The mean cluster holds 6.000, and its reciprocal — 0.167 — is the runs estimator of the extremal index, whose true value for this process is exactly 1 − 0.75 = 0.25. Every threshold method in the collection assumes exceedances are independent pieces of information; here 30 of them are 5.

The clustering the tail has

Every threshold method counts exceedances as though they were independent pieces of information, and in a dependent series they arrive in clusters. Ignoring that overstates a return level by the reciprocal of the extremal index — ×3.527 counted where the mean cluster holds four — and leaves a reported standard error 2.151 times too small.

extreme · Extremes Five treatments of an estimate above one, φ = 0.95, n = 25. The correction exceeds one on 31.1% of series at this setting. left where it lands: squared forecast error 12.828, average decay factor 0.7974 against a true 0.7351; capped at 0.995: squared forecast error 5.680, average decay factor 0.5950 against a true 0.7351; capped at 1 − 1/n: squared forecast error 5.535, average decay factor 0.5256 against a true 0.7351; correction scaled to fit: squared forecast error 5.535, average decay factor 0.5256 against a true 0.7351; correction refused where it leaves: squared forecast error 5.868, average decay factor 0.4423 against a true 0.7351.

The correction that leaves the region

The bias correction adds (1 + 3φ̂)/n whatever φ̂ is, so it pushes the estimate above one whenever φ̂ exceeds (n − 1)/(n + 3) — on 31.1% of series at φ = 0.95 and twenty-five observations. Five obvious things to do about it differ by a factor of 2.3 in squared forecast error, and none of them is documented as a choice.

evaluation · Bias Six cells, and 5% is the right answer in all of them. How often a regression between two independently generated series is called significant at the 5% level, for two worlds and three treatments, at 200 observations. Every pair is independent by construction, so 5% is correct everywhere and every other reading is a failure. Untreated: 82.9% and 100.0%. With a fitted line removed: 74.2% and 33.5%. Differenced: 5.0% and 5.2%. The treatment that controls the rate in both worlds is the one that discards the level and the trend, which is the quantity a study of trending series was about.

The repair that keeps the question

A regression between two independent trending series is significant 82.9% of the time on random walks and 100.0% on trend-stationary ones. Subtracting a fitted line leaves 74.2% and 33.5%; differencing leaves 5.0% and 5.2% and throws away the trend the study was about.

timeseries · Spurious What the forecast interval is short by, φ = 0.85, 6 steps ahead. The plug-in interval covers 88.42% against a claimed 95%. Correcting the variance recovers 0.56 points, propagating the persistence's own standard error recovers 0.40, correcting the persistence recovers 2.66, and all three together recover 4.20 — leaving 2.38 points unaccounted for.

What the interval is short by

The forecast interval covers 88.42% where it claims 95%. Correcting the persistence recovers 2.66 points, correcting the innovation variance 0.56, propagating the persistence's own standard error 0.40 — and all three together recover 4.20 of the 6.58, leaving a residual none of the standard repairs reaches.

evaluation · Bias

Named alongside it

The objects these essays reach for when they reach for this one.

AutocorrelationRandom walkMonte CarloSpurious regressionCointegrationDifferencingMean squared errorPlug in estimateUnit rootForecast errorForecast horizonBias correction

All concepts