Normal Distribution
nothing is called 95% until it has been counted
Concept

Design effect — where it appears

How many times larger a coefficient's variance is than an independent-observations calculation says, given the dependence the sample actually has. For a covariate constant within clusters of equal size it is exactly 1 + (m − 1)ρ, so the row count is not what a standard error should be dividing by.

Named by 5 essays across 3 fields — each of them below, with the objects they name alongside it.

Three answers to how much sample is left. What a set of inverse-probability weights leaves of the treated arm, by three routes, at six settings of the assignment rule. The integral 1/(π∫φ/e) reads the whole covariate space and falls from 0.9392 to 2.655e-3. Kish's effective size counted in samples of 600 falls only to 0.2861, because almost all of the integral's fall is in a region a sample of six hundred never draws from. And the fraction the variance of the weighted mean actually delivers is lower again — 0.1155 — because the variance is the average of one over the effective size and the effective size averaged is not the same number. At the widest overlap all three agree to 0.05%.

How many observations a weight leaves

Kish's effective sample size is exact — for an outcome whose mean does not move with the covariates the weights are built from, the studentised variance reads 1.0680 where the formula says one. For the population's own outcome the same reading is 6.769, rising to 52.497.

weights · Weighting The rows are held fixed; only the clusters move. Counted coverage of four 95% intervals for a slope, at five cluster counts with the row count held at 300 throughout and a within-cluster correlation of 0.1, over 6000 draws apiece, with the sizes equal. The interval that counts rows covers 53.42% at 5 clusters — a second closed form says 2Φ(z/√D) − 1 = 54.44% for a design effect of 6.900, and reads nothing about clusters at all. The cluster-robust interval read against a normal covers 74.43% there and 94.20% at 100 clusters; read against a t on G − 1 it covers 85.08% and 94.47%. The number of independent things is the cluster count, and every quantity here is blind to how many rows were typed.

The count that is not the rows

Three hundred rows in five clusters of sixty carry 6.9000 times the variance an independent-rows calculation reports, and the interval that counts rows covers 53.42%. The same five unequal sizes laid out two ways give design effects of 9.3158 and 5.4652.

sandwich · Misspecification One estimator, three answers, and only the reference changes. Coverage of the cluster-robust 95% interval for the slope against the number of clusters, at 30 rows in each. The estimator is identical in all three curves; what differs is the number it is compared against. At 5 clusters it covers 75.05% against a normal, 85.30% against a t on 4 degrees of freedom and 87.95% against a t on 3. At 80 clusters the three agree to within a point. The correction costs nothing: the same standard error, a different table.

The reference the sandwich is read against

The cluster-robust interval covers 75.05% at five clusters and 93.58% at eighty. The same estimate read against a t on G − 2 covers 87.95% at five, and the estimator is unchanged — three hundred rows grouped into five clusters cover 74.28% where the same three hundred grouped into seventy-five cover 94.63%.

sandwich · Misspecification How often the pooled test finds a relation in a panel with none, when the pairs share a shock. Against the 5% point computed for independent pairs, over 1,200 panels of pairs of 200 observations with no pair tied. Sixteen pairs, raw: 4.6% at a common share of 0, 7.2% at a common share of 0.25, 15.5% at a common share of 0.5, 27.3% at a common share of 0.75, 31.6% at a common share of 0.9; after removing the cross-sectional averages: 5.3%, 5.1%, 5.8%, 5.3%, 4.8%. Four pairs, raw: 5.4%, 6.5%, 6.8%, 12.6%, 16.9%.

A shock every pair shares

Sixteen pairs of series whose shocks are half common are worth 5.37 independent pairs to the panel's average statistic, and sixteen that share nine tenths are worth 1.44. Read against the 5% point computed for independent pairs, the pooled test then calls a panel with no relation tied 15.5% and 31.6% of the time; read against its own point, it finds a relation shared by every pair 22.3% of the time at nine tenths common, against one pair's 20.0%. Removing each date's cross-sectional average gives back the level and all sixteen pairs when every pair feels the shock equally, and leaves 13.3% false alarms when the loadings run from a half to one and a half.

cointegration · Spurious Four treatments of a shock sixteen pairs share: false alarms at the independence point, and power at each one's own point. Sixteen pairs, two hundred dates, nine tenths of each series' variation common. every pair loads the shock equally: nothing removed, false alarms 31.6% and power 22.3%; the cross-sectional average removed, false alarms 4.8% and power 99.8%; an estimated factor removed, false alarms 9.2% and power 76.3%; the observed shocks removed, false alarms 4.5% and power 6.5%. loadings run from a half to one and a half: nothing removed, false alarms 30.8% and power 25.8%; the cross-sectional average removed, false alarms 13.3% and power 82.5%; an estimated factor removed, false alarms 8.9% and power 73.8%; the observed shocks removed, false alarms 4.5% and power 7.5%.

A factor estimated from the panel

When sixteen pairs feel a common shock with loadings from a half to one and a half, removing the cross-sectional average leaves the pooled test rejecting an unrelated panel 13.3% of the time. Estimating the factor and each pair's loading instead brings that to 8.9% — and no further, because each loading comes from the same two hundred dates however many pairs there are — and it costs power against the average at the right critical value, 73.8% against 82.5%. Removing the shocks as observed restores 4.5% exactly and finds a shared relation 7.5% of the time: a series tied to its partner carries the partner's factor in its level, not its own shock.

cointegration · Spurious

Named alongside it

The objects these essays reach for when they reach for this one.

Effective sample sizeCluster-robust standard errorCluster sizeClustered samplingCointegrationCoverageCritical valueDegrees of freedomEngle–GrangerIntraclass correlationKish effective sizeSandwich estimator

All concepts