Normal Distribution
nothing is called 95% until it has been counted
Concept

Fixed-width interval — where it appears

An interval whose half-width is set in advance, so that the experiment's size rather than its precision is what the data decides. Its coverage is the expectation of 2Φ(d√N/σ) − 1 over the sample-size distribution and nothing else, which is why a random N covers less than a fixed one.

Named by 30 essays across 7 fields — each of them below, with the objects they name alongside it.

One experiment, with the blocks getting smaller as the target comes into range. A single run at a requirement of 0.25, with the block sizes 5, 5, 11, 25, 11, 8, 3, 2 and a total of 70 observations in 8 blocks. The rule stops when the observations in hand reach z²σ̂²/d², with σ̂² pooled from the within-block contrasts — an estimate that moves as the run goes on, so the target moves too. Early blocks are large because the target is far away and cannot be overshot; late ones are small because a block is the granularity of the answer. The interval afterwards is built from the 8 block means and from nothing the rule looked at, and it has 7 degrees of freedom against the rule's 62.

A block size that changes

The blinded rule's exactness never needed the blocks to be the same size. Letting the size be chosen from the contrasts as the run goes on leaves the coverage exactly where it was — and runs straight into an identity that says what a schedule can and cannot buy.

pace · Stopping The construction survives a difference of two weighted means. Coverage of δ̂ ± t√(S_D²/H) on b − 1 degrees of freedom, over 900 runs at a requirement of 0.3, where δ̂ is the block differences weighted by h_b = (1/m_A + 1/m_B)⁻¹ and H is their total. The theorem the one-mean field rests on goes through with h_b in place of the block size, and the reason is that the weights a weighted least squares decomposition needs are the inverse variances — which is exactly what h_b is. The stopping rule reads only within-arm within-block contrasts, so it is a function of nothing the interval reports, whatever it does with the block sizes. Each bar is within 2.9% of the level it claims.

A width promised for a difference

The exact fixed-width interval was built for one mean. Two arms make the target 42.7 units of effective size and each unit costs four observations, so the same promise about a difference costs 169.4 rather than 42.7 — and the theorem survives untouched with the harmonic size in place of the block size.

contrast · Width The stopping rule costs more than the weighting does. Coverage over 2000 runs of a trial whose variance ratio drifts by a factor of twenty, at three ways of deciding when to stop. Twelve blocks fixed in advance is the top line and reproduces what a trial of fixed length delivers. Stopping when the reported interval is short enough is the bottom line, and it costs between 3.0% and 5.5% of coverage — including for the rule that is told every block's true ratio, which is what says the shortfall belongs to the stopping and not to the weights. Stopping on a width predicted from the within-arm sums of squares is the middle line, and it is back at the fixed-length values. The standard error on each point is 0.49%.

A width the trial has to stop for

The weighting that covers at 94.9% on twelve blocks covers at 91.5% when the trial stops as soon as its interval is short enough — and so does the rule that is told every block's true variance ratio. The shortfall is the stopping, not the weights.

stop · Stopping Exact in the corner, where nothing was. Coverage of a nominal 95% interval on five designs, at a required half-width of 0.3. The first four are the two-arm field's own and the fifth is its corner — two variances, block sizes that swing by eight, and an allocation that alternates between five to one and one to five — where neither of that field's two conditions holds. The effective-size weights over-cover there at 98.40%; the weights h_b(λ) = (1/m_A + λ/m_B)⁻¹ cover at 94.84%, and at 94.84% when λ is estimated from the within-arm contrasts rather than known. Nothing here is supposed to move.

Weights that need only a ratio

A fixed-width interval about a difference is exact under either of two conditions and under neither in the corner. It is exact there too, and the only thing it needs is how much larger one arm's variance is than the other's.

corner · Nuisance How wrong the ratio is allowed to be. λ enters only through the weights, so misstating it leaves the estimate unbiased and moves two things — the interval's calibration and its efficiency — both of which are closed forms of the design. Coverage stays at its level over a factor of two in either direction (94.93% at half the truth, 94.27% at twice it) and starts to go at a factor of five. An estimate on hundreds of within-arm degrees of freedom is never wrong by anything like that, which is what makes the feasible rule usable rather than merely definable.

Blinded, and still exact

The one number the exact interval needs is a ratio of within-arm spreads, which is a contrast and contains no mean — so a rule forbidden to look at the effect may compute it, on more degrees of freedom than the interval itself has.

corner · Width Two promises, and no rule here keeps both. A fixed-width procedure promises two things: that the interval covers at its nominal rate, and that it is no wider than the width asked for. Over 1500 runs of the modelled weighting, a rule that stops when the interval it will report is short enough keeps the width — only 2.0% of runs come out wider than 0.34 — and covers at 91.13% against a nominal 95%. A rule that stops on a width predicted from the within-arm sums of squares covers at 94.80% and comes out wider than promised on 42.3% of runs. The two promises are in conflict because keeping the second one exactly requires conditioning on the very quantity that has to be independent of the stopping time for the first.

Stopping on the arms

The width a trial will report is predictable from quantities the interval is not about. A rule that stops on the prediction covers at 94.5% where one that stops on the interval covers at 91.5, and it costs two blocks and half of the width promise.

stop · Width No block size is best at both things the procedure claims. Two claims and one dial. The honest interval's half-width falls as the blocks get smaller, because the interval's degrees of freedom are the number of blocks: 0.2602 at blocks of two against 0.2933 at blocks of sixteen. The fixed-width claim — that the mean is within 0.25 of the truth — gets more reliable as they get larger, because the sample size is less variable: 92.40% against 95.00%. Both are computed from the same runs, and the second is reproduced to within a tenth of a point by E[2Φ(d√N/σ) − 1], which needs the sample-size distribution and nothing else. The schedules sit at the bottom left: as narrow as the smallest fixed block and as few observations, with the rule's spread estimate on half as many degrees of freedom again.

Two degrees of freedom, one total

The block size is a dial, and the two things a fixed-width procedure claims move in opposite directions along it. Divide the width by the square root of the sample size and one of them turns out to depend on the number of blocks and on nothing else.

pace · Width Three sets of weights, five designs, and no estimator that is exact everywhere. Coverage of the same interval under three weightings. h_b is the inverse variance when the arms share a variance or the allocation is constant; equal weights are right when every block has the same two counts; the estimated precision weights are right in the limit and exact nowhere, because the decomposition needs the weights to be the constants they are only estimating. In the corner — two variances, changing sizes, changing allocation — the two exact estimators are the ones that miss, at 98.45% and 95.65%, and the one with no theorem behind it is at 95.05%. That is the whole statement: there is an exact estimator under either condition, and none under both.

Which weights are the inverse variances

There is an exact estimator when the two arms share a variance and another when every block has the same two counts, and between them they cover every trial anybody designs on purpose. In the corner where neither holds, both cover 98.45% instead of 95%, and the only estimator at its level is the one with no theorem behind it.

contrast · Nuisance One rule keeps its promise and the other keeps its budget. Both stopping rules at five requirements, 1,500 experiments each, with a first stage of 5. The upper curve is the two-stage rule: 97.1%, 96.2%, 95.9%, 96.1%, 96.0% — at or above 95% at every point, which is a theorem rather than a tendency, because its interval is built from a spread estimated before the stopping point was chosen. It pays 2.06×, 2.01×, 2.00×, 1.99×, 1.99× the observations that knowing σ would need. The lower curve is the rule that re-estimates after every observation: 94.5%, 89.3%, 91.0%, 91.5%, 94.3%, on 0.98×, 0.87×, 0.88×, 0.93×, 0.96×. The second rule is the one anybody would run and the first is the one whose claim is true.

Stopping when it is precise enough

An experiment that runs until its estimate is precise enough is the natural design and the one with a theorem against it. Its two-stage cousin keeps its promise exactly, for every unknown spread, and pays twice the observations for it.

guarantee · Stopping The weights may not read the block they weight. A weighted least squares decomposition needs weights that are constants, or at least independent of the differences they multiply. One λ̂ pooled across the trial is estimated on hundreds of degrees of freedom and is effectively a constant; a λ̂ estimated inside each block is estimated on that block's own two or three, and is correlated with the difference it weights. Coverage falls from 94.68% to 82.76% — and the interval gets wider while doing it, 0.5163 against 0.3024, which is the signature of weights that are noise.

The condition that cannot be dropped

The weights may not read the block they weight. Estimate the variance ratio inside each block rather than across the trial and the coverage falls to 83% — on an interval that is at the same time seventy per cent wider.

corner · Allocation Two arms leave one degree of freedom per block unaccounted for. Each point is one run. The one-mean field's identity is (b − 1) + (N − b) = N − 1, and every schedule moves along that line rather than off it. Two arms give the rule N − 2b and the interval b − 1, which come to N − b − 1 — short of the N − 2 two arms leave by exactly one per block, since a block's arm counts absorb one degree of freedom each and only one of the two directions carries the difference. The hollow points add what the block sums are worth, b − 1 more, and land on the total. The missing degrees of freedom are not lost; they are in a place the interval has to be shown it may read.

The degrees of freedom in the sums

One arm partitions N − 1 exactly. Two arms give the rule N − 2b and the interval b − 1, which is short by one per block — and the missing ones are in the block sums, which are correlated with the differences at −0.79 and are usable anyway.

contrast · Blocking Exact coverage, at every block size. Coverage of the interval each rule reports, at a nominal 95%, over 2,500 runs each with a standard error of 0.44 points. The blinded rule stops on the within-block contrasts and reports an interval built from the block means, and those two are independent whatever the rule does — so the interval is an ordinary t interval on b − 1 degrees of freedom and its coverage is exact. It is exact at every block size drawn. The interval a practitioner writes at the purely sequential rule's stopping time covers 91.72%, and Stein's two-stage rule is exact for the same reason as the blinded rule and spends 2.10 times the observations to be so. The bars are truncated at 86% so the differences can be seen.

The rule that cannot see the mean

A sequential rule stops when its own estimate of the spread is small, which is more often on the samples whose spread came out low — so the interval afterwards is short. There is a way to keep updating the estimate and stop being able to see the mean at all.

blind · Stopping The overshoot is the last block size and nothing else. A run stops at a multiple of its own block sizes and cannot land between them, so it ends past its own target by about half a block. Fixed sizes overshoot by 1.5, 2.2, 3.1, 4.7, 8.5 observations as the size goes 2, 3, 5, 8, 16. Every schedule here ends in blocks of two and every one of them lands where blocks of two land — 1.62, 1.32, 1.37 against 1.48 — while having spent most of the run inside blocks four and eight times larger. That is the one thing on this page a schedule genuinely takes from both ends.

What a schedule actually buys

Big blocks early and small blocks late is the right instinct and it does not take both ends of the trade, because there are not two ends to take. What it does take is the overshoot — about four per cent of the observations — and a steadier stopping point.

pace · Nuisance What a fixed-width interval covers, by the number of blocks the trial ran before it stopped. Two thousand runs of each rule, the modelled weighting, a promise of 0.34. Reading its report: 4–8 blocks, 22.3% of runs, 78.2%; 9–12 blocks, 16.6% of runs, 90.4%; 13–16 blocks, 18.4% of runs, 96.2%; 17–20 blocks, 17.4% of runs, 96.0%; 21–28 blocks, 17.9% of runs, 96.4%; 29–36 blocks, 7.4% of runs, 99.3% — 91.45% overall. Reading the arms: 4–8 blocks, 0.0%, none; 9–12 blocks, 0.9%, 94.4%; 13–16 blocks, 30.4%, 95.6%; 17–20 blocks, 50.0%, 93.9%; 21–28 blocks, 18.0%, 94.4%; 29–36 blocks, 0.7%, 92.3% — 94.50% overall.

The trials that stopped early

A fixed-width trial that stops when its own interval is short enough covers 91.45% — an average of 78.2% among the 22.3% of runs that stop within eight blocks and 96% to 99% among those that run longer. Widening every interval by 17.1% brings the average to 95% and leaves the early stops at 85.6%, while 92.8% of runs now report an interval wider than the width they promised. Even doubling every interval leaves the early stops short.

stop · Width A wrong weight costs width; a random weight costs level. Five weightings on a trial whose variance ratio drifts by a factor of 20.1 between the first block and the last, over 4000 runs. The rule that knows every λ_b covers at 95.1% and sets the width. One ratio for the whole trial is wrong for every block and costs nothing in level — 94.8% — while being 20% wider; equal weights are calibrated by an identity and 22% wider. The ratio estimated inside each block is the only rule aimed at the quantity that actually varies, and it is the only one that misses the level, at 92.0%: a weight computed from a handful of degrees of freedom is mostly noise, and noise in a weight is not a wrong weight. Modelling the drift across blocks recovers the oracle's width at 94.8%.

A ratio that changes between blocks

A wrong weight costs width and a random weight costs level. The rule aimed at the quantity that actually varies is the only one that misses its own coverage, and the rule that models it across blocks recovers the whole of what knowing it is worth.

blocks · Nuisance What a schedule is allowed to read, and what happens when it reads more. The construction allows the block sizes to be anything at all as long as they are functions of the within-block contrasts, which are independent of every block mean. A schedule that shrinks the block whenever the between-block spread is running above what the contrasts say is a direct attempt to hold down the quantity the interval will be built from, and it succeeds: the estimate lands at 0.8373σ² against the honest 0.9831, and the coverage goes with it. Growing the block on the same spread instead pushes the other way and over-covers — which is not a repair, it is the same violation with the sign reversed, and the level is no longer a property of the procedure at all.

A schedule that reads the mean

The block sizes may be anything at all provided they are functions of the contrasts. Two natural schedules break that, in opposite directions — and the most natural mistake of the three is not a schedule at all but a stopping rule, at 86.87% coverage and fewer observations.

pace · Stopping Four intervals at one stopping time. 3,500 experiments under the sequential rule with a first stage of 5 and a required half-width of 0.4, which is a demand that knowing σ would meet with 24.0 observations and which the rule meets with 20.8. The rule's own interval covers 90.3%. Replacing the fixed width by a t interval on the same data gives 92.0%. Keeping the rule's own random sample size and drawing a fresh sample of that size gives 89.8% at the fixed width and 95.5% for a t interval — so the sample size being random costs nothing, and the sample size being chosen by the data the interval is built from costs the rest. The spread estimated at the stopping moment is 17.1% below the truth, which is the same fact one level down.

The interval after a stop it chose

A rule that stops when the estimated precision is good enough stops on the samples whose estimate was small. Its interval covers 90% and claims 95%, and a fresh sample of the same random size covers 95.4%.

guarantee · Stopping Free until the sums stop seeing what the differences see. Coverage with and without the block sums pooled into the interval's variance estimate. With one effect and one level they are free. With an effect that varies between blocks they are still free, because a block's sum picks that variation up exactly as its difference does. With a level that varies they make the interval 37% wider and conservative. And where the effect falls as the level rises — a ceiling, and not an exotic thing to suppose — the sums carry none of the between-block variation while the differences carry all of it, the pooled estimate is short, and the interval that uses it covers 88.75% on a width 20% narrower than the honest one.

What a two-arm rule may not pool

A spread computed "within the block" without the arm label carries a share of the effect, so the trial runs 173 observations at a null and 282 at an effect of 1.5. The stopping rule is reading the thing it exists to measure, and the phrase that produced it is one word long.

contrast · Allocation What the exactness costs, and the dial it is bought with. The median half-width of the interval each rule reports, at a requirement of 0.4 and a first look after 5 observations. The flat line is the interval a practitioner writes at the purely sequential rule's stopping time, which covers 91.72% rather than 95%. The curve is the blinded rule, which covers its nominal level at every block size: it reads b − 1 degrees of freedom where the other reads n − 1, and pays for the exactness in width. The best block size is 3, at 0.4712. Larger blocks give the stopping rule a better estimate and the interval a worse one, and the two costs go opposite ways, which is what puts the minimum in the middle.

What the blindfold costs

The exactly-covering rule pays for it in the width of the interval, and the block size is a dial between two costs that run in opposite directions. And on an interval whose width was fixed in advance, the same repair buys nothing at all.

blind · Nuisance The fixed-width trial's coverage when the outcomes are not normal, for both stopping rules. normal: stopping on the arms 94.05% after 18.1 blocks, on the report 89.95%; log-normal, skewness 0.95: stopping on the arms 94.70% after 18.5 blocks, on the report 90.80%; log-normal, skewness 2.26: stopping on the arms 94.15% after 19.3 blocks, on the report 90.25%; log-normal, skewness 4.75: stopping on the arms 94.45% after 18.7 blocks, on the report 90.50%; t, five degrees of freedom: stopping on the arms 94.35% after 18.3 blocks, on the report 90.30%; skewness 4.75, arm A only: stopping on the arms 93.80% after 26.0 blocks, on the report 89.90%; skewness 4.75, arm B only: stopping on the arms 93.60% after 14.2 blocks, on the report 89.95%; equal variances, normal: stopping on the arms 94.75% after 11.4 blocks, on the report 90.90%; equal variances, skewness 4.75: stopping on the arms 94.05% after 11.1 blocks, on the report 92.00%.

A width rule on skewed outcomes

The blinded fixed-width rule rests on a within-arm spread being independent of the arm means, which only normal samples guarantee. On outcomes with a skewness of 4.75 the independence fails and the overall coverage barely notices — 93.60% to 94.70% across every shape counted, against 94.05% on normal outcomes. What skew moves is the runs that stop by twelve blocks, which cover about 90% with the skew in one arm, and the trial's length: a variance ratio corrected on normal theory lengthens it from 18.1 blocks to 26.0 with the skew in the first arm and shortens it to 14.2 with the skew in the second.

stop · Width One pooled ratio, two periods. The calibration factor of weights built from one variance ratio pooled across the trial — the factor by which the interval's own scale estimate overstates the estimate's variance — when arm B's ratio is F times higher in one kind of period than the other, closed form on the corner's block compositions. With the small A-heavy blocks the noisier, the factor rises: 1.00 at F = 1, 1.50 at F = 1.5, 2.07 at F = 2, 3.12 at F = 3, 3.91 at F = 4, 4.86 at F = 6, 5.44 at F = 8. With the large B-heavy blocks the noisier, it falls: 1.00 at F = 1, 0.74 at F = 1.5, 0.65 at F = 2, 0.58 at F = 3, 0.55 at F = 4, 0.53 at F = 6, 0.53 at F = 8. A ratio estimated per period gives weights that are the inverse variances again, and its factor is exactly 1 at every F.

A ratio that changes with the period

The corner's exact interval needs one variance ratio, and the corner's own alternating allocation makes two kinds of period that need not share one. When they differ the pooled ratio is precise and wrong: at a fourfold difference the interval covers 100.0% one way round and 85.1% the other. A ratio estimated per period is exact again, blind to the effect, and costs under a tenth of the width — and a ratio that merely wanders at random costs width and no coverage at all.

corner · Nuisance Where the bias lands. The drift in the log variance ratio, fitted across 12 blocks over 4000 trials. E[log λ̂_b] is log λ_b plus ψ(k_B/2) − log(k_B/2) − ψ(k_A/2) + log(k_A/2), which depends on nothing but the degrees of freedom — so the tempting sentence is that it goes into the intercept and leaves the slope alone. It does not, because the blocks alternate between allocations and the alternation is correlated with the covariate being fitted: the lopsided blocks carry 0.5383 of bias and the even ones carry none. Uncorrected the slope reads 1.5597 against a truth of 1.5, which is 8.0 standard errors. Subtracting the two digammas block by block leaves 1.4976.

The bias that lands in the slope

The bias in a log variance estimate depends on nothing but its degrees of freedom, so it goes into the intercept — unless the degrees of freedom alternate with the design, which is exactly what a block-randomised trial makes them do.

blocks · Width A lean, its reflection, and none. The mean error of each weighting's estimated difference at a fixed eighteen blocks, 2,000 trials a shape, with two standard errors. normal: the modelled ratio, every block 0.0031, the modelled ratio, each block left out 0.0028, the modelled ratio, cross-fitted halves 0.0023, the true ratios 0.0037; skewness 0.95: the modelled ratio, every block 0.0092, the modelled ratio, each block left out -0.0134, the modelled ratio, cross-fitted halves -0.0014, the true ratios 0.0002; skewness 2.26: the modelled ratio, every block 0.0166, the modelled ratio, each block left out -0.0298, the modelled ratio, cross-fitted halves -0.0042, the true ratios -0.0029; skewness 4.75: the modelled ratio, every block 0.0255, the modelled ratio, each block left out -0.0455, the modelled ratio, cross-fitted halves -0.0055, the true ratios -0.0051.

A weighting that does not read its own spreads

On skewed outcomes a block's spreads move with its means, so a weighting fitted to every block's spreads leans: +0.0255 at a skewness of 4.75. The obvious repair — fit each block's weight with that block left out — does not remove the lean. It reverses it and nearly doubles it, to −0.0455, because a weighted mean divides by the sum of the weights and every other block's weight still reads the one left out. Splitting the trial into two halves and weighting each by the other's fit does remove it: −0.0055, indistinguishable from the weighting told the true ratios, for a spread about a tenth wider.

stop · Width How much of the forbidden spread a stopping rule may read. The rule stops when the observations in hand reach z²ŝ²/d², with ŝ² a blend: a share 1 − κ of the pooled within-block estimate it may read, and a share κ of the between-block estimate the interval is built from, which it may not. At κ = 0 it is the honest rule; at κ = 1 it stops on the reported spread. Each curve is the coverage lost against κ = 0 on the same 6,000 runs, with two paired standard errors, at three requirements. At a half-width of 0.35 the loss is 0.87% at κ = 0.2 and peaks at 3.97%. At 0.25 it is 0.20% at 0.2, 0.98% at 0.5 and 5.50% at 1. At 0.15 nothing measurable is lost until κ = 0.9, and 3.73% at 1. The cost of reading is not proportional to how much is read; it is concentrated where the rule leans on the forbidden spread, and a longer experiment can lean further before paying.

How much of the forbidden spread a rule may read

A stopping rule that reads the spread its interval will report breaks the interval. Blend that spread into what the rule reads, a share at a time, and the cost is not proportional to the share — at a tight requirement a rule can read four fifths of it before anything is lost, and the last fifth costs nearly four points.

pace · Stopping A third stage, between the two rules. Four rules for one requirement: a half-width of 0.4 at 95%, which knowing σ would meet with 24.0 observations, after a first stage of 5. Each bar is the mean number of observations as a multiple of that, and the note beside it is the coverage. The two-stage rule spends 2.03× and covers 95.42%, computed exactly. The three-stage rule — a second stage to half of what the first spread asks for, the spread re-read on everything, a third stage to the rest — spends 0.86× and covers 88.42%; given 8.50 more observations in every experiment's last stage it covers 95.00% on 1.20×. The rule that re-estimates after every observation spends 0.88× and covers 90.5% over 4,000 drawn experiments; it is the only row that has to be drawn, because it is the only one whose stopping point shares observations with the mean.

The fifth a third stage never reaches

A three-stage stopping rule keeps the two-stage rule's independence and the sequential rule's budget. With a first stage of five it covers 88.4% — worse than the rule it was meant to repair — because in a fifth of experiments the second stage buys nothing.

guarantee · Stopping The sums, pooled as they are and made independent. Coverage on four designs at a requirement of 0.9, 20,000 runs each, about 3.0, 3.6, 3.4, 3.0 blocks. Refusing the sums: 94.91%, 95.06%, 94.96%, 94.86%. Pooling them as they are: 94.98%, 93.67%, 96.17%, 94.93% — exact at balance, short when the allocation is unbalanced and the same in every block, long when it alternates. Pooling the residualised sums, each block's sum less the part of it its own difference predicts: 94.98%, 95.16%, 95.06%, 94.70%, on every design. Each bar is ±0.31%.

The sums made independent

The block sums are a free second copy of a two-arm trial's variance only at balance. With ten and two in every block, pooling them covers 93.67% on four blocks; swap the arms each block and it covers 96.17%. Take from each sum the part its own difference predicts and the copy is independent at every allocation.

contrast · Blocking What each stopping rule spends. The average number of observations the corner design takes before it stops, with the stopping rule reading pooled within-arm spreads (the upper bar) and each kind of period's own spreads (the lower), at a promised half-width of 0.3, 3,000 trials each. one ratio: 1,656 and 1,649; 2-fold, small blocks noisier: 1,020 and 1,015; 2-fold, large blocks noisier: 2,976 and 2,614; 4-fold, small blocks noisier: 751 and 669; 4-fold, large blocks noisier: 5,642 and 3,368; 8-fold, small blocks noisier: 711 and 490; 8-fold, large blocks noisier: 10,989 and 3,019. With one ratio the two rules are the same rule. With the large blocks noisier the pooled rule over-spends by a factor that grows with the difference, to 3.6 times at eightfold; with the small blocks noisier it over-spends less.

A stopping rule that reads the periods

A fixed-width trial in the corner stops when its estimated precision reaches the promise, and it estimated that precision from spreads pooled across a trial whose two kinds of period need not share a variance ratio. Read each kind of period's own spreads instead and the trial stops on 3,019 observations where it took 10,989, at an eightfold difference, with the interval it reports still covering 95.1%. The stopping time reads nothing but within-arm spreads, per period now as well as pooled, and that is all the exactness needed.

corner · Nuisance Three repairs at one requirement. Coverage against spend at a half-width of 0.4 and a first stage of 5, all exact. A floor on the second stage at 5, 8, 10, 12, 15, 18, 21, 24 observations covers 88.4%, 91.4%, 92.4%, 93.1%, 93.9%, 94.6%, 95.3%, 96.0% at 0.86, 0.94, 0.97, 1.00, 1.02, 1.05, 1.09, 1.14 times the oracle's spend. The second stage reading with t, with its fraction dialled from 0.2 to 1, covers 88.1%, 89.0%, 90.6%, 92.1%, 93.4%, 94.4%, 95.2%, 95.9%, 96.4% at 0.84, 0.91, 1.02, 1.17, 1.34, 1.52, 1.70, 1.89, 2.08 times. A constant at the end covers 88.4%, 89.4%, 90.4%, 91.3%, 92.1%, 92.9%, 93.6%, 94.2%, 95.0%, 95.7% at 0.86, 0.90, 0.94, 0.98, 1.02, 1.06, 1.10, 1.14, 1.20, 1.26 times.

A floor that reads the first stage

A three-stage stopping rule fails at a first stage of five because a low first reading asks for nothing more, and the only floors that repair it were set near the answer, by somebody who already knew roughly what the spread was. Inflating the first reading by Stein's own factor needs no such knowledge, and it does cut the experiments left stuck from a fifth to a fifteenth. But those still stuck cover less than before, the rule needs a different dial at every requirement, and a floor of 26 observations covers 96.5% at the spend where the rule covers 92.1%, even met by a spread it was not set for.

guarantee · Stopping Coverage as the effect moves with the level. Coverage on ten and two in every block, with the level between blocks moving at a spread of 0.5 and the effect moving with it at each slope, 4000 runs each. At -3: refusing the sums 94.4%, pooling them 88.8%, residualised 97.0%; At -2.5: refusing the sums 94.3%, pooling them 88.9%, residualised 95.6%; At -2: refusing the sums 94.9%, pooling them 90.0%, residualised 93.8%; At -1.5: refusing the sums 95.1%, pooling them 91.7%, residualised 92.7%; At -1: refusing the sums 95.1%, pooling them 94.8%, residualised 94.0%; At -0.5: refusing the sums 95.1%, pooling them 97.2%, residualised 97.7%; At 0: refusing the sums 95.0%, pooling them 98.8%, residualised 99.6%; At 0.5: refusing the sums 95.0%, pooling them 99.4%, residualised 99.9%; At 1: refusing the sums 95.0%, pooling them 99.4%, residualised 99.9%.

The sums under a moving level

A trial's block sums are a free second estimate of its variance only while they see what the differences see, and a level that moves between blocks with an effect moving against it breaks that: pooled raw at a ceiling they cover 87.6% to 90.0%. Residualising the sums on an unbalanced design does not make them safe. It moves the failure. On ten and two, 90.0% at the ceiling becomes 93.8%, the trough leaves for a slope of about −1.5, and over the rest of the range the same estimate over-covers, at 99.9% on an interval half as wide again as it needs to be.

contrast · Blocking Coverage lost against the honest rule. Coverage lost in points against the rule that reads only within-block spreads, on the same observations, blocks of 2, 6000 runs, at half-widths 0.5, 0.35, 0.25, 0.15. the reported spread in full: 0.32 ± 0.27, 2.53 ± 0.37, 5.33 ± 0.41, 3.75 ± 0.35. a fixed half: 0.72 ± 0.26, 2.43 ± 0.29, 0.98 ± 0.26, -0.25 ± 0.20. κ = (b − 1)/(N − 1): 0.37 ± 0.21, 1.18 ± 0.25, 0.58 ± 0.23, -0.18 ± 0.19. the same, phased in as b/(b + 20): 0.22 ± 0.10, 0.35 ± 0.16, 0.23 ± 0.18, -0.10 ± 0.19. the same, read in only after twenty blocks: 0.00 ± 0.00, -0.02 ± 0.04, 0.17 ± 0.19, -0.17 ± 0.19.

A rule that sets its own κ

A stopping rule may read the spread inside its blocks and may not read the spread between them, and the dial between the two costs nothing for a long stretch and then a lot. The share an efficiency argument recommends is each estimate's share of the degrees of freedom, and on blocks of two at a half-width of 0.15 that loses nothing and steadies the experiment by 30%. At 0.35 it loses 1.18 points. Deferring the forbidden reading until twenty blocks costs nothing measurable at any requirement, and it keeps most of the steadiness where steadiness can be had.

pace · Stopping

Named alongside it

The objects these essays reach for when they reach for this one.

CoverageDegrees of freedomBlindingStopping ruleBlockingMonte CarloVariance ratioConfidence intervalEfficiencyNuisance parameterSample sizeWeighted least squares

All concepts