Normal Distribution
nothing is called 95% until it has been counted
Concept

Kish effective size — where it appears

The number of equally weighted observations a weighted sample is worth, computed as the square of the summed weights over the summed squared weights. It is a statement about the weights alone, and it is the variance of a weighted mean only when the outcome does not move with them.

Named by 4 essays across 2 fields — each of them below, with the objects they name alongside it.

Three answers to how much sample is left. What a set of inverse-probability weights leaves of the treated arm, by three routes, at six settings of the assignment rule. The integral 1/(π∫φ/e) reads the whole covariate space and falls from 0.9392 to 2.655e-3. Kish's effective size counted in samples of 600 falls only to 0.2861, because almost all of the integral's fall is in a region a sample of six hundred never draws from. And the fraction the variance of the weighted mean actually delivers is lower again — 0.1155 — because the variance is the average of one over the effective size and the effective size averaged is not the same number. At the widest overlap all three agree to 0.05%.

How many observations a weight leaves

Kish's effective sample size is exact — for an outcome whose mean does not move with the covariates the weights are built from, the studentised variance reads 1.0680 where the formula says one. For the population's own outcome the same reading is 6.769, rising to 52.497.

weights · Weighting The estimator has no upper bound on what it costs. What a thinning overlap does to a stabilised inverse-probability estimate of an average effect of 1.0000, over 600 samples of 600 at each of six settings. The spread rises from 0.1965 to 0.8122 and the root mean square error from 0.1964 to 0.9219, so at the thin end the error is very nearly the whole of the quantity being estimated. The lower line is the share of the arm's weighted total the single largest observation owns, averaged over the same draws: 0.69% to 9.78%, and in the worst single draw of the sweep 82.75%. Coverage of the 95% interval goes from 93.7% to 55.5%.

The region with no comparison

A trimmed interval covers the average effect over everybody 90.8% of the time at six hundred rows and 41.0% at nine thousand six hundred, while covering the average effect over the units it kept 94.3% and 96.0% throughout. An interval that gets worse as the sample grows is an interval about something else.

weights · Weighting The rows are held fixed; only the clusters move. Counted coverage of four 95% intervals for a slope, at five cluster counts with the row count held at 300 throughout and a within-cluster correlation of 0.1, over 6000 draws apiece, with the sizes equal. The interval that counts rows covers 53.42% at 5 clusters — a second closed form says 2Φ(z/√D) − 1 = 54.44% for a design effect of 6.900, and reads nothing about clusters at all. The cluster-robust interval read against a normal covers 74.43% there and 94.20% at 100 clusters; read against a t on G − 1 it covers 85.08% and 94.47%. The number of independent things is the cluster count, and every quantity here is blind to how many rows were typed.

The count that is not the rows

Three hundred rows in five clusters of sixty carry 6.9000 times the variance an independent-rows calculation reports, and the interval that counts rows covers 53.42%. The same five unequal sizes laid out two ways give design effects of 9.3158 and 5.4652.

sandwich · Misspecification Coverage of the cluster-robust slope interval at six, ten and twenty clusters of unequal size, by reference. 5,000 studies of 300 rows at each setting, within-cluster correlation 0.2. 6 clusters, equal sizes: 89.1%, 92.6%, 96.0%; 6 clusters, 5:1, big at the edges: 80.1%, 88.1%, 95.2%; 6 clusters, 5:1, big at the centre: 93.8%, 95.4%, 96.8%; 6 clusters, 15:1, big at the edges: 76.2%, 86.8%, 94.5%; 10 clusters, equal sizes: 91.1%, 93.0%, 95.8%; 10 clusters, 5:1, big at the edges: 86.3%, 90.3%, 94.7%; 10 clusters, 5:1, big at the centre: 94.5%, 95.7%, 96.4%; 10 clusters, 15:1, big at the edges: 84.9%, 89.6%, 94.8%; 20 clusters, equal sizes: 92.9%, 93.8%, 95.1%; 20 clusters, 5:1, big at the edges: 90.9%, 92.7%, 94.8%; 20 clusters, 5:1, big at the centre: 94.5%, 95.0%, 95.6%; 20 clusters, 15:1, big at the edges: 90.6%, 92.4%, 94.7% — for the usual variance on G − 2, the bias-reduced variance on G − 2, and the bias-reduced variance on its effective count.

An effective number of clusters

Ten clusters of thirty rows, a covariate spread evenly across them, and a cluster-robust interval for the slope read against a t on G − 2 = 8 covers 91.1%. The slope needed about five degrees of freedom, not eight, because the two outermost clusters carry 49% of its variance and the middle two 0.6%. A count computed from the sizes and positions before any outcome is seen — 4.45 here, 3.32 when the big clusters sit at the edges — with the bias-reduced variance covers between 94.5% and 96.8% in all twelve layouts measured, where the usual interval on G − 2 covers between 76.2% and 94.5%.

sandwich · Misspecification

Named alongside it

The objects these essays reach for when they reach for this one.

Effective sample sizeCluster-robust standard errorCluster sizeClustered samplingConfidence intervalCoverageDegrees of freedomDesign effectExtreme weightIntraclass correlationInverse-probability weightingLeverage

All concepts