Normal Distribution
nothing is called 95% until it has been counted
Concept

The Pocock boundary — where it appears

A group-sequential boundary held at one constant value across every look, raised just enough that the looks together spend the stated error rate. It stops more trials early than O'Brien–Fleming and pays with a higher bar at the final analysis.

Named by 4 essays across one field — each of them below, with the objects they name alongside it.

Four boundaries for 5 looks, all spending 5% in total. test at 0.05 every look: 1.96, 1.96, 1.96, 1.96, 1.96. Pocock — a constant, higher boundary: 2.41, 2.41, 2.41, 2.41, 2.41. O'Brien–Fleming — strict early, nearly nominal at the end: 4.55, 3.22, 2.63, 2.27, 2.03. Bonferroni across looks: 2.58, 2.58, 2.58, 2.58, 2.58. Every one except the first spends the same total error rate; they differ in when they spend it.

Spending the error rate

The repair for interim testing is to spend 5% across the looks rather than at each one. The boundaries are solvable rather than quotable, and a trial that can stop early uses 298 observations where a fixed design uses 400 — at a cost of half a point of power.

sequential · Stopping Forty O'Brien–Fleming trials at a true effect of 0.16, with the boundary written as an effect. The dashed line is the smallest effect a trial can report and still stop at each look: 0.510 at 80 observations, 0.255 at 160 observations, 0.170 at 240 observations, 0.128 at 320 observations, 0.102 at 400 observations. The true effect is 0.16, so at 3 of the five looks a trial cannot stop without reporting more than it. 29 of these forty trials stop before the last look, each marked where it stopped.

The effect a stopped trial reports

An O'Brien–Fleming trial at 88.45% power holds its error rate exactly and reports an effect 9.6% too large on average. The 11.39% of trials that stop at the second look report 1.83 times the truth, the ones that cross at the last look report 0.80 times it, and pooling every trial by its size gives the truth back to the last digit.

sequential · Stopping The chance of crossing later from each interim z, under six schedules with O'Brien–Fleming-type spending boundaries. Exact. At an interim |z| of 1.0: end only 3.64%, +0.6 3.41%, +0.75 3.20%, +0.9 3.41%, every 0.125 3.00%, every 0.05 2.88%. At 2.5: end only 38.30%, +0.6 45.80%, +0.75 45.36%, +0.9 41.70%, every 0.125 50.08%, every 0.05 53.24%. The heavy line is the largest of the six at each z.

A look the trend asked for

Under an O'Brien–Fleming-type spending function, every schedule of looks fixed in advance spends exactly 5.0000%. A committee that adds a look at three quarters of the trial whenever the interim z is 1.5 or more spends 5.2323% — 5.315% counted over a hundred thousand trials — and the most a committee choosing among six schedules could spend is 5.4390%.

sequential · Stopping Power against expected observations for two fixed schedules and a look added when the interim trend is close, O'Brien–Fleming-type spending. Straight to the end: power 89.01% at 338.9 expected observations. A look at three quarters always: 88.38% at 300.7. Adding the look when the interim |z| is at least 0.5, 1, 1.5, 2, 2.5, with boundaries corrected to spend exactly 5%: 88.41% at 302.0; 88.45% at 302.8; 88.56% at 306.3; 88.76% at 315.5; 88.95% at 330.1. With the spending boundaries alone the same rules spend 5.059%, 5.156%, 5.232%, 5.187%, 5.058%.

A look added by a rule

A monitoring committee's rule — add a look at three quarters of the trial when the interim |z| is at least 1.5 — spends 5.2323% under O'Brien–Fleming-type boundaries. Written into the protocol it can be given boundaries that spend exactly 5%, three ways: raise the added look's boundaries by 2.05%, raise every later boundary by 0.99%, or preserve at each interim value the chance of crossing the plan would have given. All three cost the same, 0.38 points of power, from 88.94% to 88.56%, because the power the uncorrected rule seemed to keep was the excess error. Made exact, the rule is a design between the two fixed schedules — and at every trigger slightly better than randomising between them.

sequential · Stopping

Named alongside it

The objects these essays reach for when they reach for this one.

Group-sequential designO'Brien–FlemingAlpha spendingClosed formError rateInterim analysisStatistical powerStopping ruleMonte CarloAdaptive designConditional distributionOptional stopping

All concepts