The observations that repeat each other

Every interval, test and standard error elsewhere on this site divides by the square root of the sample size. That division is not arithmetic; it is a claim — that each observation contributes its own information, uncorrelated with what the others contribute.

Where the observations arrive in time order, the claim is usually false, and the failure is not a rounding error.

Fifty observations, worth six #

The process here is the simplest one that has memory: each observation is φ times the previous one plus fresh noise. At φ = 0.8, each value carries eight tenths of its predecessor forward.

Nothing about that makes any individual observation less informative than an independent one. What it does is make the observations redundant with each other, and the redundancy has a closed form. The variance of the sample mean is inflated by

(1 + φ)/(1 − φ)

which is 9 at φ = 0.8. Turned around, the fifty correlated observations estimate the mean about as precisely as n(1 − φ)/(1 + φ) = 5.6 independent ones.

That number is the field in one line. Fifty observations that look like fifty observations, arrive like fifty observations, and fill fifty rows of a dataset, are worth six.

What the interval does #

A 95% interval for the mean, built the ordinary way, from series of fifty at φ = 0.8, covers the true mean 47.0% of the time.

The same code at φ = 0 covers 94.6%, which is the calibration that makes the first number mean something. Nothing about the interval is broken. It is being handed a standard error computed as if the fifty values were independent, and they are not.

The failure is one-directional and that is worth stating, because it makes the error worse than a random one. Positive dependence — the usual kind, where consecutive values resemble each other — always inflates the variance of the mean and therefore always makes the interval too narrow. There is no configuration in which ignoring positive autocorrelation makes an analysis conservative.

The asymptotic form is not the number at fifty #

The inflation (1 + φ)/(1 − φ) is a limit, and a fifty-point series is not the limit. The finite version sums the autocovariances that actually fit inside the series:

1 + 2 Σ (1 − k/n) φᵏ

which at n = 50 and φ = 0.8 is 8.20 rather than 9.

The distinction is not pedantry. A check written against the asymptotic form and measured against a well-run simulation is a tenth out, which is comfortably enough to fail — and the natural conclusion from a failing check is that the account is wrong rather than that the formula was the wrong one. The site’s own gate is written against the finite sum for that reason, and it agrees with the measured spread of the sample mean to within a few per cent at every φ tested.

The sample variance is biased too, in the same direction #

Underneath the interval there is a second error working the same way, and it is the more surprising of the two.

For any stationary process, the expected sample variance is

E[s²] = (n/(n−1)) · (γ₀ − Var(x̄))

and Var(x̄) is exactly the quantity dependence inflates. So positive autocorrelation makes s² too small: a correlated series looks less variable than the process that generated it.

Both errors therefore point the same way. The estimate of the spread is too small, and the divisor √n claims more information than exists, so the interval is narrowed twice over by two mechanisms that are easy to mistake for one.

This is also what rescues an apparently broken measurement in the differencing essay: a variance ratio measured at 0.674 against an asymptotic prediction of 0.600 looks like a 12% error, and is the same finite-sample bias acting in opposite directions on a positively correlated series and its negatively correlated differences. Putting it in makes the two routes agree to four digits.

The 47% is reproducible from the two biases together #

The two failures — the mean’s variance being inflated and the sample variance being deflated — are described in separate sections, and multiplying them reproduces the headline coverage exactly. That is worth doing, because it says there is no third thing going wrong.

At n = 50 and φ = 0.8 the finite-sample inflation of the mean’s variance is

$$\frac{1+\varphi}{1-\varphi} - \frac{2\varphi\left(1-\varphi^{n}\right)}{n(1-\varphi)^{2}} = 9 - 0.8 = 8.20,$$

so the true standard deviation of the sample mean is $\sigma\sqrt{8.20/50} = 0.405\sigma$.

The sample variance is short by the same quantity divided by n: $\operatorname{E}[s^2] = \sigma^2(1 - 8.20/50) = 0.836\sigma^2$, so the standard error the interval is built from averages $\sigma\sqrt{0.836/50} = 0.1293\sigma$.

The interval is $\pm 1.96$ of the second and has to cover a quantity whose spread is the first, so it covers about

$$2\Phi\!\left(\frac{1.96 \times 0.1293}{0.405}\right) - 1 = 2\Phi(0.626) - 1 = 46.9\% .$$

Against 47.0% measured on the simulation.

Which of the two does how much #

Having the arithmetic in one place also splits the blame, and the split is not even.

The variance the interval should be using is 8.20 times the one it computes from independent-sample arithmetic. The variance it does compute is a further 16.4% low, because $s^2$ is biased down. The two multiply to 9.81 — the interval’s implied variance is understated by nearly a factor of ten, of which 8.2 is the mean’s own inflation and the remaining 1.20 is the biased scale estimate.

So the redundancy of the observations is about six sevenths of the problem and the biased variance estimate is about one seventh. That ordering is worth having because the two are usually presented as one complaint, and only the first is the effect an effective sample size is about.

It also says what a partial repair buys. Correcting the variance estimate alone and leaving the inflation untouched moves the coverage from 47.0% to about 50.6% — under four points, on an interval claiming ninety-five. Correcting the inflation alone and leaving $s^2$ as it is gets to 92.7%. Neither is a fix; the first is barely a change.

The repair, and what it does not do #

The correction that follows from the inflation is to divide by the effective sample size instead of the actual one: estimate φ from the series, form n(1 − φ̂)/(1 + φ̂), and build the interval with that.

It works, mostly. Coverage at φ = 0.8 goes from 47.0% to 89.1% — a long way back towards 95% and not all the way, because φ̂ is itself estimated from the same short series and is biased downward, which leaves the interval slightly too narrow.

That residual gap is the honest state of the repair, and it points at the general rule for this field: a correction estimated from the same dependent data cannot fully fix the problem the dependence causes. More sophisticated versions — a Newey–West standard error, a block bootstrap, an explicit model of the process — improve on it and share the property that they need enough data to estimate the dependence structure, which is exactly what a short dependent series does not have.

Where this arises, which is nearly everywhere #

The vocabulary of this field is about time, and the structure appears wherever observations are ordered by anything.

Repeated measurements on the same unit. Two readings from the same patient, plot, machine or customer are correlated, and a dataset of a thousand readings from a hundred units carries far less than a thousand observations’ worth of information. That is the same arithmetic as groups that borrow from each other, arrived at from the opposite direction: there the correlation within a group was the thing to be estimated, here it is the thing that makes the naive standard error wrong.

Spatial data. Neighbouring plots, adjacent postcodes, nearby sensors. The correlation is by distance rather than by lag and everything above transfers.

Anything sampled faster than it changes. A sensor reading every second on a process that drifts over minutes produces observations that are almost copies of each other, and the effective sample size can be orders of magnitude below the row count. Sampling more often does not help; it adds rows without adding information.

And clustered survey data. Households within a village, pupils within a school, patients within a practice. The design effect a survey statistician quotes is exactly the variance inflation factor above, computed from within-cluster correlation instead of from a time lag.

What the effective sample size is not #

The phrase invites two readings and only one of them is right, so it is worth separating them.

It is not a claim that some observations are useless. All fifty values are used, all fifty contribute to the estimate, and discarding forty-four of them to leave “the six that count” would produce a worse estimate than using all fifty. The mean of the fifty is the right estimate; what is wrong is the advertised precision of it.

And it is not a fixed property of the dataset. It is specific to the quantity being estimated. The effective sample size for the mean is n(1 − φ)/(1 + φ); for the variance it is a different number; for a regression slope against another series it is different again. A dataset does not have one effective size, it has one per question.

The second point has a practical consequence that is easy to miss. Software that reports a “design effect” or an effective sample size is reporting it for a particular estimand, usually a mean or a total, and carrying that number over to a different analysis of the same data is not valid.

Thinning does not help, and shows why #

A tempting repair is to keep every fifth observation, on the grounds that widely spaced values are nearly independent, and then use the ordinary formulas on what is left.

It does work in the sense that the interval built from the thinned series covers correctly — the retained observations really are nearly independent at φ⁵ = 0.33, and nearly so at φ¹⁰ = 0.11. It is still the wrong thing to do, because the estimate from ten thinned values is noisier than the estimate from all fifty.

The full series carries more information than its effective size suggests for the mean, since the discarded values do contribute, just less than independent ones would. Thinning throws that contribution away in exchange for an easier formula.

The general form of the error is worth recognising because it recurs: making the data fit the method is almost always worse than using a method that fits the data. The same instinct produces dropping the censored subjects when the tool wants a single number, and it costs thirty-one percentage points there.

One series looks like a trend #

There is a second failure here that no standard error addresses, and it is about reading rather than computing.

A single realisation of a strongly correlated process wanders. It drifts up for a while, then down, and it produces stretches that look exactly like trends, cycles and regime changes. Nothing in the generating rule contains a trend, a cycle or a regime — it is one line of arithmetic with a single parameter — and any individual series will still support a confident narrative about what happened in the middle third.

This is why every figure in this field draws twenty series rather than one. Twenty of them make it obvious that the wandering is the process rather than an event, and one of them makes it obvious that something happened. A reader shown a single series cannot correct for this by being told to be careful; the correction has to be in the picture.

Negative dependence, which nobody warns about #

Everything so far assumes φ is positive, which is the ordinary case and not the only one.

At negative φ, consecutive observations alternate: a high value tends to be followed by a low one. The same formula applies and the inflation factor (1 + φ)/(1 − φ) falls below one — at φ = −0.5 it is a third, so fifty alternating observations are worth about a hundred and fifty independent ones.

The interval is then too wide rather than too narrow, and the analysis is conservative. That is why nobody warns about it: the error is in the safe direction and it costs power rather than credibility.

Negative dependence is rarer in raw data and extremely common in processed data, because differencing creates it — a differenced series of independent observations has a lag-one correlation of exactly −0.5. Any pipeline that differences by habit has installed alternation into data that did not have it, and the resulting series is not the series anyone thinks they are analysing.

What has to be true for any of this #

The measurements above assume the process is stationary: its mean, variance and correlation structure do not change over the series. That is a real restriction and it is the dividing line between this essay and the next.

A stationary series has a mean to estimate. It wanders around that mean and returns to it, the autocorrelations decay geometrically, and the effective sample size is a sensible quantity — smaller than n, and proportional to it.

A random walk does not have any of those properties. It has no mean to return to, its variance grows without limit, and no amount of data pins down a centre, because there is not one. Everything in this essay stops applying, and the failures get considerably worse: the next essay measures a regression between two independent random walks rejecting the true null of no relationship 76.7% of the time.

The practical version of the distinction is worth carrying: a series that returns to a level can be summarised by that level; a series that does not, cannot. Most economic and financial series are the second kind, most physical measurements around a controlled setpoint are the first, and the two are not always distinguishable by eye.

Reading a reported standard error #

The arithmetic here converts into one question a reader can ask of any analysis of ordered data, and it is worth stating in the form that gets an answer.

How many independent observations does this dataset contain?

Not how many rows. A study reporting a mean from a thousand sensor readings taken a second apart on a process that drifts over an hour has a row count of a thousand and an effective size that may be in the tens. A survey of two thousand pupils in forty schools has an effective size closer to the number of schools than to the number of pupils, whenever the thing measured varies more between schools than within them.

If the analysis has not asked the question, its standard errors are the ones computed from the row count, and the direction of the error is known: too small, by a factor whose square root is the inflation above. At φ = 0.8 that factor is nearly three on the standard error itself, which turns a result reported at p = 0.001 into one at about p = 0.1.

That last conversion is the reason this field is not a footnote about intervals. The dependence does not shift a conclusion slightly; it moves results across the threshold that decides whether they are reported at all.

What this field will establish #

Four essays.

This one is the arithmetic of dependence: the variance of the mean inflates by a factor that has a closed form, fifty observations at φ = 0.8 are worth about six, and the ordinary 95% interval covers 47%.

The next is the same failure applied to a relationship rather than to a mean, where it stops being a matter of intervals being too narrow and becomes a matter of findings that are not there — 76.7% of pairs of unrelated series called significant, with a median R² that any journal would print.

Then what differencing costs, which is the standard repair, works, and is not free: it takes the false-positive rate from 76.7% to 4.9% and takes most of a real relationship with it.

And last, the check to run before trusting any standard error — its size, its power, and the range of dependence it will miss, which at φ = 0.2 is four times out of five.

All four rest on the same generated truth. The processes here are two lines of arithmetic with a single parameter, so what any procedure should have reported is known exactly, and every rate quoted is a count of how often it reported something else.