Normal Distribution
nothing is called 95% until it has been counted
Concept

Alpha spending — where it appears

A rule giving how much of the error rate may have been spent by each fraction of the information collected. Each look's boundary is solved from what the rule allows by then, so the looks may fall wherever they happen, provided their timing is not chosen from the data.

Named by 4 essays across one field — each of them below, with the objects they name alongside it.

Four boundaries for 5 looks, all spending 5% in total. test at 0.05 every look: 1.96, 1.96, 1.96, 1.96, 1.96. Pocock — a constant, higher boundary: 2.41, 2.41, 2.41, 2.41, 2.41. O'Brien–Fleming — strict early, nearly nominal at the end: 4.55, 3.22, 2.63, 2.27, 2.03. Bonferroni across looks: 2.58, 2.58, 2.58, 2.58, 2.58. Every one except the first spends the same total error rate; they differ in when they spend it.

Spending the error rate

The repair for interim testing is to spend 5% across the looks rather than at each one. The boundaries are solvable rather than quotable, and a trial that can stop early uses 298 observations where a fixed design uses 400 — at a cost of half a point of power.

sequential · Stopping The chance of crossing later from each interim z, under six schedules with O'Brien–Fleming-type spending boundaries. Exact. At an interim |z| of 1.0: end only 3.64%, +0.6 3.41%, +0.75 3.20%, +0.9 3.41%, every 0.125 3.00%, every 0.05 2.88%. At 2.5: end only 38.30%, +0.6 45.80%, +0.75 45.36%, +0.9 41.70%, every 0.125 50.08%, every 0.05 53.24%. The heavy line is the largest of the six at each z.

A look the trend asked for

Under an O'Brien–Fleming-type spending function, every schedule of looks fixed in advance spends exactly 5.0000%. A committee that adds a look at three quarters of the trial whenever the interim z is 1.5 or more spends 5.2323% — 5.315% counted over a hundred thousand trials — and the most a committee choosing among six schedules could spend is 5.4390%.

sequential · Stopping Power against expected observations for two fixed schedules and a look added when the interim trend is close, O'Brien–Fleming-type spending. Straight to the end: power 89.01% at 338.9 expected observations. A look at three quarters always: 88.38% at 300.7. Adding the look when the interim |z| is at least 0.5, 1, 1.5, 2, 2.5, with boundaries corrected to spend exactly 5%: 88.41% at 302.0; 88.45% at 302.8; 88.56% at 306.3; 88.76% at 315.5; 88.95% at 330.1. With the spending boundaries alone the same rules spend 5.059%, 5.156%, 5.232%, 5.187%, 5.058%.

A look added by a rule

A monitoring committee's rule — add a look at three quarters of the trial when the interim |z| is at least 1.5 — spends 5.2323% under O'Brien–Fleming-type boundaries. Written into the protocol it can be given boundaries that spend exactly 5%, three ways: raise the added look's boundaries by 2.05%, raise every later boundary by 0.99%, or preserve at each interim value the chance of crossing the plan would have given. All three cost the same, 0.38 points of power, from 88.94% to 88.56%, because the power the uncorrected rule seemed to keep was the excess error. Made exact, the rule is a design between the two fixed schedules — and at every trigger slightly better than randomising between them.

sequential · Stopping How much a look added at three quarters spends above 5%, by what the decision to add it reads. A two-look O'Brien–Fleming-type trial, interim at half the information, adds a look at three quarters with a stated chance under the null. Its error rate above 5%, in points, at chances of 0.1, 0.3, 0.5, 0.7, 0.9: triggered by a calendar or by the within-arm variance, 0.00000, 0.00000, 0.00000, 0.00000, 0.00000; by the blinded variance with 20 patients at the interim, 0.034, 0.064, 0.069, 0.054, 0.022; with 100, 0.014, 0.027, 0.030, 0.025, 0.012; with 1,000, 0.004, 0.008, 0.009, 0.008, 0.004; by the interim trend itself, 0.234, 0.163, 0.089, 0.042, 0.013. Exact integrals over the interim statistic.

The variance that contains the effect

A monitoring committee that adds an interim look because the variance came in higher than planned is reading a nuisance quantity, and a look added on a nuisance quantity should leave the spending function's 5% alone. It does, exactly, when the variance is the within-arm one, and when the trigger is the calendar. It does not when the variance is the one computed with the arm labels hidden: that variance is the within-arm sum of squares plus the interim statistic squared, so it is high more often when the trend is strong. The excess is 0.069 points at a twenty-patient interim and falls as one over the square root of the interim's size — a tenth of what a trigger on the trend itself spends, and never zero.

sequential · Stopping

Named alongside it

The objects these essays reach for when they reach for this one.

Error rateGroup-sequential designInterim analysisO'Brien–FlemingThe Pocock boundaryStatistical powerStopping ruleClosed formAdaptive designBlinded analysisGroup-sequentialMonte Carlo

All concepts