Counting what is still wandering

A spectrum is not a count. Three eigenvalues at 0.2519, 0.0276 and 0.0064 look like one large and two small, but “looks large” is not a decision procedure, and the whole point of putting a number on something is to stop deciding by eye.

Turning the spectrum into an integer needs a statistic and a threshold. The statistic is easy. The threshold is the interesting half, and it is where this field repeats — one dimension worse — the situation the pair’s residual test was already in.

The statistic #

Take the eigenvalues in descending order. To test whether the rank is at most q, pool everything below the cut:

trace(q) = −T · Σ log(1 − λᵢ) for i > q

Each term is small when its eigenvalue is small and grows when it is not, so the sum is large exactly when the eigenvalues below the cut are larger than a system of rank q would produce. Multiplying by T, the number of usable observations, is what puts it on a scale that does not shrink with the sample.

There is a second statistic, and the two answer different questions. The maximum-eigenvalue statistic uses only the next eigenvalue along, testing rank exactly q against rank exactly q + 1. The trace pools everything below the cut, testing rank at most q against rank greater than q. Neither dominates: the trace has more power against several small departures at once and the maximum-eigenvalue statistic has more against one large one. Both are computed here and the sequential procedure below uses the trace, because the pooling matches what the chain of decisions is actually asking at each step.

The three statistics, from the three eigenvalues #

The statistic pools everything below a rank, so one system’s spectrum produces one number per decision, and it is worth computing them once to see what the chain is actually comparing.

With $-n\sum_{i>q}\log(1-\lambda_i)$ at three hundred observations and eigenvalues 0.2519, 0.0276 and 0.0064:

• rank ≤ 0, pooling all three: 97.4
• rank ≤ 1, pooling the bottom two: 10.3
• rank ≤ 2, the smallest alone: 1.9

Three numbers, and each one is compared against a different null distribution — which is the whole difficulty this essay is about, and is invisible when the statistic is described in the singular.

One property of the last of them is worth noticing because it is what makes any fixed threshold legitimate. For a small eigenvalue $-n\log(1-\lambda) \approx n\lambda$, and the nuisance eigenvalues fall like 1/n, so $n\lambda$ settles to a constant as the sample grows: 1.9 here, and about 8.0 for the second eigenvalue at every length. The statistic for the spurious directions has a limiting distribution that does not depend on n at all, which is exactly the condition a table of critical values needs and is not something the eigenvalues themselves have.

Three distributions, one name #

Here is the part that makes this a field rather than a footnote.

The distribution of the trace statistic under its null does not depend on the true rank of the system, on the adjustment speeds, on the variances of the shocks, or on the length of the series. It depends on exactly one thing: k − q, the number of series still wandering freely under the null being tested.

That is a strong claim and it is the claim the whole apparatus rests on, because every critical value used anywhere in this field is applied to a system it was not simulated from. So it is asserted rather than assumed: the 5% point for one common trend comes out at essentially the same value from series of length 200 and length 300, and the 5% point for three common trends is somewhere else entirely.

Simulated on three thousand systems at n = 300, the 5% points are:

• 8.12 with one common trend,
• 18.64 with two,
• 31.74 with three.

And the 1% points are 12.01, 23.69 and 38.05.

None of those is a chi-square quantile, and none of them is available from any table this site contains. They are simulated on one set of seeds and applied on another, which is the same discipline the pair’s residual test needed for the same reason: the statistic is a functional of a random walk rather than a sum of squared normals, so the limiting distribution is not in the family the tables are built from.

What happens if the wrong row is used #

An assertion that has never rejected anything proves nothing, and the same goes for a critical value that has never been shown to matter. So the machinery is fed the mistake it is most likely to make.

The mistake is not exotic. The statistic is called the trace statistic in all three cases; a table lookup keyed on “the trace test at the 5% level” is a plausible thing to write; and a three-series system tested for rank at most 0 against the value for one common trend is off by a factor of nearly four.

Read against 31.74, which is the right row for a three-series system with nothing in it, the test rejects at close to its nominal 5%. Read against 8.12, which is the row for a single common trend, it calls three unrelated random walks cointegrated most of the time. The failure is total and the run that produces it looks exactly like the run that does not: same code, same statistic, same printed output, a number compared against a number.

The chain #

One test gives a yes or no. A count needs a chain of them.

Test whether the rank is at most 0. If that fails to reject, stop and report 0. If it rejects, test whether the rank is at most 1, against the critical value for two common trends rather than three. If that fails to reject, stop and report 1. And so on.

Each link uses a different distribution, because each link leaves a different number of series wandering. That is why the critical values above are a table with one row per trend count rather than a single number: the sequential procedure walks down the table as it goes, and a procedure that used one row throughout would be making the mistake of the previous section at every step but the first.

What the statistics actually look like #

Numbers help here, because the gap between the statistic and its threshold turns out to be much wider than the gap between the eigenvalues that produced it.

Averaged over two hundred systems at n = 300, the three trace statistics come out as:

• on rank-zero systems, 21.6, 7.9 and 1.5, against thresholds of 31.74, 18.64 and 8.12 — every one below its own row, so the chain stops immediately and reports 0;
• on rank-one systems, 97.3, 10.3 and 1.9 — the first clears 31.74 by a factor of three, the second is well under 18.64, and the chain stops after one step and reports 1;
• on rank-two systems, 182.9, 61.4 and 3.1 — the first two clear their rows decisively and the third does not.

The three chains that produce those numbers are the whole procedure, and one of them ends after a single comparison while another runs to three. That is worth noticing next to the essay on choosing a left-hand side: the two-step procedure performs exactly one test whatever the truth is, and the count of tests performed here is itself part of the answer.

What is worth noticing is the asymmetry between the two ends. The statistic that should clear its threshold does so by an enormous margin, and the one that should not falls short by a comfortable one. The decision is not close in either direction, which is why the accuracy is in the mid-nineties rather than scraping along at chance — and it is why the errors that remain come from the tails rather than from genuine ambiguity.

The margin comes from the −T factor. An eigenvalue of 0.2519 contributes about 0.29 to the sum, which is not a large number; multiplied by nearly three hundred observations it becomes 87. So the statistic inherits its scale from the sample size, which is exactly what makes the threshold constant in n: as the series get longer, the eigenvalues belonging to trends shrink like 1/n and their contributions to the sum stay put, while the eigenvalue belonging to a real relation stays put and its contribution grows without limit. The null distribution does not move and the alternative walks away from it.

Both errors, counted separately #

At n = 300 with five hundred systems of each kind, the procedure returns the true rank:

• 95.0% of the time on rank-zero systems,
• 96.2% on rank-one systems,
• 95.8% on rank-two systems.

Those three numbers are close together and it would be easy to report them as “about 96% accurate” and stop. That would hide the thing worth knowing, which is that the residual is made of two errors with completely different characters.

Under-counting is a power problem and it goes away. At n = 80 the procedure under-counts a rank-one system 18.4% of the time: the relation is there and the series are too short to see it. At n = 150 that is 0.2%. At n = 300 and beyond it is 0.0%. This error behaves the way an error is supposed to — more data removes it.

Over-counting is the test’s own level and it does not. It is 4.4% at n = 80, 4.4% at n = 150, 3.8% at n = 300, 5.8% at n = 600 and 3.8% at n = 1000. Those numbers are bouncing around 5% and going nowhere, because they are 5% — the chance that a test at the 5% level rejects a true null. The chain stops at the first non-rejection, so it will over-shoot about one time in twenty at any length, and the only thing that changes the number is changing the level.

This is a shape the site has met before in a different dress. In spending the error rate the quantity that does not improve with more data is the familywise rate; here it is the top of a sequential chain. In both cases the residual error is not a shortcoming of the procedure but a specification of it, and reporting the two errors together disguises a number that is chosen as a number that is achieved.

What the count is worth #

The count is not an end in itself, so it is worth measuring what having it buys against the alternative a cautious analyst would reach for.

The alternative is differencing. Every series here is a random walk, regressions between random walks are untrustworthy, and the standard repair is to work with the changes instead. It is a good repair for the problem it addresses and it destroys the information this field is about.

On rank-one systems the rank procedure finds a relation 100.0% of the time and a regression on the differenced series finds one 31.2% of the time. On rank-two systems the numbers are 100.0% and 42.2%. And on systems with nothing in them neither invents anything: 4.6% and 3.0%, both at a nominal 5%.

So the safe move costs about two thirds of the findings that are there, and it does so silently: a differenced regression that finds nothing produces the same output as a differenced regression on a system with nothing in it. The analyst who differenced has not been warned, because there is nothing in the arithmetic to warn them with.

What is being assumed, said out loud #

Three things, because a procedure that reports an integer with no standard error should be explicit about what it is standing on.

The critical values are simulated for this specification. A constant unrestricted in the error-correction equations, no deterministic trend, no short-run lags. Change any of those and the null distribution moves, and every published table for this test is indexed by exactly those choices for that reason. The values here are this site’s own, generated by the same code that uses them.

The chain’s level is the level of each link, not of the whole. Testing at 5% at every step does not make the procedure a 5% procedure in any familywise sense, and the over-counting rate measured above is the visible consequence of that.

And the count is a count of relations in the model as written. Three series that are genuinely tied by something non-linear, or tied at a lag this specification does not include, will be counted as untied — correctly, in the sense that no linear combination of the levels is stationary, and misleadingly if anybody reads “rank 0” as “unrelated”.

That third one deserves more than a clause, because it is the assumption a reader is most likely to forget and the one this site’s habits are least able to police. Every figure in this field is generated from a mechanism of the form Δy = Πy₋₁ + ε, so every system tested here is one the model can describe. That makes the measurements honest about the procedure and silent about the model: an accuracy of 96.2% is 96.2% given that the truth is in the family being searched, and no simulation in which the analyst writes both the generator and the estimator can say anything about the case where it is not.

The same limitation applies to every counted rate on this site and it is worth stating in the field where the estimate is an integer, because an integer looks more like a fact than a coefficient does. “Rank 1” is shorthand for “one dimension of the space of linear combinations of these three levels, at this lag specification, with this deterministic term, comes out stationary”. It is not shorthand for “these series are related in one way”.

Reading it without the machinery #

Not everyone who meets three wandering series is going to run a reduced-rank decomposition, so it is worth saying what the eigenvalues alone are good for.

The ratio is more informative than any single value. At rank one the average spectrum is 0.2519, 0.0276 and 0.0064: the first is nine times the second, and the second is four times the third. At rank zero it is 0.0446, 0.0212 and 0.0050 — ratios of two and four, with nothing standing out. A gap of an order of magnitude between consecutive eigenvalues is the thing to look for, and it is visible before any threshold is applied.

A large eigenvalue has a meaning that does not need a table. It is a squared canonical correlation, so 0.2519 says there is a combination of the levels accounting for a quarter of the variance of some combination of the changes. That is a statement a reader can weigh against what they know about the series, which is more than a p-value on its own offers.

And the smallest eigenvalue is a diagnostic in its own right. If every eigenvalue is large, the model is wrong: full rank means every series is stationary, and these were not. Seeing 0.0064 at the bottom is confirmation that at least one thing in the system genuinely wanders, which is the premise the whole procedure is built on.

The check, and the refusal that makes it mean something #

The claim gated here is the one the field depends on and cannot see from inside a figure: that the null distribution is a function of the trend count and of nothing else. This field’s library asserts it in both directions — the same trend count at two sample sizes gives the same 5% point, and a different trend count gives one more than twice as large.

The refusal is the wrong row of the table, run deliberately. Three unrelated walks, tested for rank at most zero, against the critical value for one common trend: the rejection rate goes from about 5% to most of them, which is what makes the right row worth indexing carefully. And the second refusal is the one every field on this site ends with — a system with nothing in it must not produce a finding. It does not: the rank procedure returns zero 95.0% of the time on three unrelated walks, and the rest is the 5% that was asked for.