Normal Distribution
nothing is called 95% until it has been counted
Series

Spurious — the series

11 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. Two independent random walks, 100 steps. Nothing connects these two series: each is generated from its own independent draws. Regressing one on the other gives a slope with t = -10.9, R² = 0.55 and p = 0.0e+0 — a result that would be reported as a finding by any standard output.

    Two walks and a finding

    Regress one random walk on another, independently generated, and the slope is significant 76.7% of the time with a median R² of 0.17. Nothing connects the two series, nothing in the output says so, and more data makes it worse.

    part 1 · timeseries
  2. What differencing fixes, and what it costs, 100 steps. The first pair is the false-positive rate for two independent random walks: 77% on the levels, 4.9% on the differences. The second pair is how much of a real relationship survives: R² falls from 0.91 to 0.33. The same operation does both.

    What differencing costs

    Differencing takes the false-positive rate between two unrelated walks from 76.7% to 4.9%, and takes a genuine relationship's R² from 0.91 to 0.33. Applied to a series that did not need it, it doubles the variance and installs a correlation of −0.5 that the data never had.

    part 2 · timeseries
  3. A pair pulled back at 20% of the gap per step. Above, the two series. Below, the difference between them. The gap is pulled back towards zero by 20% of itself each step, so it stays inside a band of 14.3 while the series themselves travel much further. Nothing here is stationary except the difference. The faint line below is the gap for two free walks from the same seed, drawn for comparison.

    The regression that is not spurious

    Two random walks regressed on each other are called significantly related three times in four, so the time-series field ends in a warning. The exception it names and does not measure is here — and when the pair is genuinely tied, the fitted relation converges at rate 1/n rather than the usual 1/√n.

    part 3 · cointegration
  4. Where the residual test's statistic actually falls, at n = 200. Four thousand pairs of unrelated random walks, each regressed on the other and each residual tested for a unit root. The statistic is computed as a t and its distribution is not a t: five per cent of it falls below -3.38, where the ordinary one-sided 5% point of a t on 198 degrees of freedom is -1.65. Everything left of -1.65 — 70.2% of the whole distribution — is a pair of unrelated walks that a t table calls cointegrated.

    The test with no table

    The statistic that separates a real long-run relation from a spurious one is computed as a t and is not a t. At two hundred observations its 5% point is −3.38 where the t table says −1.65, and reading it against the table calls two unrelated random walks cointegrated 70.5% of the time.

    part 4 · cointegration
  5. The damage and the warning, against the same dial. Two readings at each persistence. In the darker colour, how often a regression between two independent series of 200 steps is called significant at 5%: 4.9% at φ = 0, 34.2% at 0.8, 52.4% at 0.9, 83.4% at a unit root. In the lighter, how often the standard unit-root test refuses a unit root on one of those series — the chance the analyst is told the series is stationary and may be regressed: 87.2% at φ = 0.9 and 31.9% at 0.95. At φ = 0.9 both are high at once, which is a correct diagnostic licensing a regression that is wrong half the time.

    The cliff that is a slope

    A regression between two independent series is called significant 4.9% of the time at no persistence, 52.4% at a lag-one correlation of 0.9, and 83.4% at a unit root. The rule the field offers asks whether the last of those holds, and at 0.9 the unit-root test correctly refuses one 87.2% of the time.

    part 5 · timeseries
  6. Six cells, and 5% is the right answer in all of them. How often a regression between two independently generated series is called significant at the 5% level, for two worlds and three treatments, at 200 observations. Every pair is independent by construction, so 5% is correct everywhere and every other reading is a failure. Untreated: 82.9% and 100.0%. With a fitted line removed: 74.2% and 33.5%. Differenced: 5.0% and 5.2%. The treatment that controls the rate in both worlds is the one that discards the level and the trend, which is the quantity a study of trending series was about.

    The repair that keeps the question

    A regression between two independent trending series is significant 82.9% of the time on random walks and 100.0% on trend-stationary ones. Subtracting a fitted line leaves 74.2% and 33.5%; differencing leaves 5.0% and 5.2% and throws away the trend the study was about.

    part 6 · timeseries
  7. How fast a gap has to close before a sample can see it close. The power of the test against the half-life of a disagreement, at 100, 200, 400 observations, each read against its own simulated critical value. Every pair in every reading is genuinely tied together, so a non-rejection is a miss. At 200 observations a gap that halves in 3 steps is found 99.9% of the time and one that halves in 12 steps is found 15.3% of the time — and by 35 steps the reading is 6.1%, which is the test's own size. Beyond that the curves are flat because there is nothing left to detect with.

    How slow a return a sample can see

    At two hundred observations the test finds a gap that halves in five steps four times in five, one that halves in eight 37.3% of the time, and one that halves in fifty 4.95% of the time — which is the rate at which it finds pairs with no mechanism at all. The boundary moves with the sample, not with its square root.

    part 7 · timeseries
  8. How slow a return each test can see at 200 observations: the relation estimated against the relation supplied. The estimated relation's test reaches 80% power up to a half-life of 5.05 steps and the supplied relation's up to 6.88. At a half-life of 8 the two find the pair 36.3% and 71.9% of the time; at 12, 16.9% and 39.3%. Their 5% points are −3.38 and −2.89.

    A relation supplied in advance

    When theory supplies the long-run coefficient, the gap between two series can be tested directly and the search the Engle–Granger test pays for disappears: its 5% point moves from −3.38 to −2.89, and at two hundred observations the slowest return found four times in five goes from a half-life of 5.05 steps to 6.88. The price is in what a rejection means. With a gap that halves in three steps, a coefficient a fifth too large is confirmed 99.1% of the time and one half again too large 81.9%, though every such gap contains a random walk — and at a thousand observations the second is confirmed more often, not less.

    part 8 · cointegration
  9. How slow a shared return a panel of pairs can see at 200 observations each. The average of the pairs' Engle–Granger statistics, against its own 5% point. One pair reaches 80% power up to a half-life of 5.1 steps, four pairs 9.2, sixteen 15.0. At a half-life of 12 the three find the relation 20.0%, 54.2% and 99.8% of the time.

    A slow return across many pairs

    One pair of two hundred observations is found tied four times in five only if its gap halves within 5.1 steps. Sixteen independent pairs sharing the same half-life, tested by the average of their Engle–Granger statistics, are found four times in five up to a half-life of 15.0 — and at a half-life of twelve, 99.8% of the time against one pair's 20.0%. What the widened test rejects is a statement about the panel: with half the pairs tied it rejects 55.4% of the time, with a quarter 21.2%, and a pair-by-pair reading of a panel in which all sixteen are tied finds 2.65 of them, against 0.78 when none are.

    part 9 · cointegration
  10. How often the pooled test finds a relation in a panel with none, when the pairs share a shock. Against the 5% point computed for independent pairs, over 1,200 panels of pairs of 200 observations with no pair tied. Sixteen pairs, raw: 4.6% at a common share of 0, 7.2% at a common share of 0.25, 15.5% at a common share of 0.5, 27.3% at a common share of 0.75, 31.6% at a common share of 0.9; after removing the cross-sectional averages: 5.3%, 5.1%, 5.8%, 5.3%, 4.8%. Four pairs, raw: 5.4%, 6.5%, 6.8%, 12.6%, 16.9%.

    A shock every pair shares

    Sixteen pairs of series whose shocks are half common are worth 5.37 independent pairs to the panel's average statistic, and sixteen that share nine tenths are worth 1.44. Read against the 5% point computed for independent pairs, the pooled test then calls a panel with no relation tied 15.5% and 31.6% of the time; read against its own point, it finds a relation shared by every pair 22.3% of the time at nine tenths common, against one pair's 20.0%. Removing each date's cross-sectional average gives back the level and all sixteen pairs when every pair feels the shock equally, and leaves 13.3% false alarms when the loadings run from a half to one and a half.

    part 10 · cointegration
  11. Four treatments of a shock sixteen pairs share: false alarms at the independence point, and power at each one's own point. Sixteen pairs, two hundred dates, nine tenths of each series' variation common. every pair loads the shock equally: nothing removed, false alarms 31.6% and power 22.3%; the cross-sectional average removed, false alarms 4.8% and power 99.8%; an estimated factor removed, false alarms 9.2% and power 76.3%; the observed shocks removed, false alarms 4.5% and power 6.5%. loadings run from a half to one and a half: nothing removed, false alarms 30.8% and power 25.8%; the cross-sectional average removed, false alarms 13.3% and power 82.5%; an estimated factor removed, false alarms 8.9% and power 73.8%; the observed shocks removed, false alarms 4.5% and power 7.5%.

    A factor estimated from the panel

    When sixteen pairs feel a common shock with loadings from a half to one and a half, removing the cross-sectional average leaves the pooled test rejecting an unrelated panel 13.3% of the time. Estimating the factor and each pair's loading instead brings that to 8.9% — and no further, because each loading comes from the same two hundred dates however many pairs there are — and it costs power against the average at the right critical value, 73.8% against 82.5%. Removing the shocks as observed restores 4.5% exactly and finds a shared relation 7.5% of the time: a series tied to its partner carries the partner's factor in its level, not its own shock.

    part 11 · cointegration

All series