Confidence interval — where it appears

A block size that changes

The blinded rule's exactness never needed the blocks to be the same size. Letting the size be chosen from the contrasts as the run goes on leaves the coverage exactly where it was — and runs straight into an identity that says what a schedule can and cannot buy.

pace · Stopping

A width promised for a difference

The exact fixed-width interval was built for one mean. Two arms make the target 42.7 units of effective size and each unit costs four observations, so the same promise about a difference costs 169.4 rather than 42.7 — and the theorem survives untouched with the harmonic size in place of the block size.

contrast · Width

An interval that carries its scale

A percentile interval inherits the resampled distribution's skewness and its scale error together. The standard repair is one extra variance per resample. It was named and not run, so this runs it.

student · Bootstrap

The instrument and the reading

Every comparison between two block windows in this collection is an error in an implied long-run variance. Nobody reads a long-run variance. Read on the 95% point a test uses, the same bootstrap costs half as much again.

readout · Bootstrap

What the 95% refers to

An interval that claims 95% is making a checkable statement about a procedure, not about the interval in front of you. Build every possible sample and count, and the interval taught first turns out to cover 87.6% of the time.

intervals · Coverage

Three mechanisms and one dataset

Four rules for which outcomes go missing, each calibrated to lose the same 35% of the rows and each leaning on what it reads with the same coefficient. Three leave the fitted slope exactly where it was, and the one that reads the outcome moves it by 0.163531.

missing · Missingness

Five times in six

A 95% interval is read as a 95% chance that a replication's estimate will land inside it. With the spread known and a replication of the same size, the chance is 83.42% — five times in six — because both estimates are uncertain. An original that landed two standard errors from the truth captures a replication 48.40% of the time; a replication a tenth the size lands inside 44.54% of the time; and among significant originals from studies with 17% power, 66.94%.

alongside · Repetition

Two degrees of freedom, one total

The block size is a dial, and the two things a fixed-width procedure claims move in opposite directions along it. Divide the width by the square root of the sample size and one of them turns out to depend on the number of blocks and on nothing else.

pace · Width

What studentising costs

Averaged over eight cells the studentised interval is 2.09 times as wide as the percentile one and covers 0.46 points better. At the block lengths the rules choose, the scale it divides by rests on two or three numbers.

student · Bootstrap

Which weights are the inverse variances

There is an exact estimator when the two arms share a variance and another when every block has the same two counts, and between them they cover every trial anybody designs on purpose. In the corner where neither holds, both cover 98.45% instead of 95%, and the only estimator at its level is the one with no theorem behind it.

contrast · Nuisance

Dropping the incomplete rows

Push the missingness until the rows that survive have a covariate mean of 0.543905 against a population zero and a variance of 0.5041 against one, and the fitted slope is still exactly right. Where the rule reads the outcome instead, the same sweep takes coverage to 2.42% at eight hundred rows.

missing · Missingness

A tenth as wide, and both of them right

The interval for a mean and the interval for one future observation are both labelled 95%, and at a hundred observations one is 10.05 times the other — exactly the square root of n + 1. Read the narrow one as the wide one and it covers a new value 15.7% of the time.

estimated · Bands

Two intervals that overlap

Two 95% intervals that just touch are read as a difference at the edge of significance. With equal standard errors their difference has p = 0.0056, not 0.05; two intervals can overlap by 58.6% of an arm and still differ at exactly 5%; standard-error bars that just touch mark p = 0.157; and when the two estimates are correlated at 0.8, touching intervals conceal a difference of 6.2 standard errors. Read as a test, non-overlap needs 1.66 times the sample for the same power.

alongside · Repetition

The degrees of freedom in the sums

One arm partitions N − 1 exactly. Two arms give the rule N − 2b and the interval b − 1, which is short by one per block — and the missing ones are in the block sums, which are correlated with the differences at −0.79 and are usable anyway.

contrast · Blocking

The ordering reverses again

One field found two of four rules changing sign between two readings of one resampling. Turn the same resamples into a studentised interval instead of a percentile one and all four change sign.

student · Bootstrap

Twenty intervals and one expected miss

The 95% belongs to the procedure, not to the interval in front of you. Twenty intervals from twenty samples make that visible in a way no definition does, and the one that misses is not a mistake.

intervals · Repetition

Where the two schools agree

With a flat prior on a normal mean, the credible interval and the confidence interval are the same interval, endpoint for endpoint. Knowing exactly when that stops being true is more useful than either camp's general argument.

bayes · Credible

One imputation is not an observation

Three ways of filling a missing outcome, under a mechanism that makes dropping the rows beyond reproach. Filling with the observed mean covers 13.85%, filling with a fitted value covers 80.85%, adding noise covers 85.78%, and the thing all three were meant to improve on covers 95.93%.

missing · Missingness

Robust is not free

A robust standard error's promise is asymptotic and its use is not. Its 95% interval covers 88.73% at twenty rows, and under mild heteroskedasticity it is the worse of the two intervals until a hundred.

sandwich · Misspecification

The region with no comparison

A trimmed interval covers the average effect over everybody 90.8% of the time at six hundred rows and 41.0% at nine thousand six hundred, while covering the average effect over the units it kept 94.3% and 96.0% throughout. An interval that gets worse as the sample grows is an interval about something else.

weights · Weighting

The interval at the end of the curve

The interval most software prints around a survival curve covers 89.7% at five years, where 3.3 of forty subjects are still being watched and where the curve is actually read. The same variance carried on a log–log scale covers 94.8% there — and the failure was never the width.

survival · Censoring

A coverage table with its own error

Twenty cells estimating the coverage of an interval that is exactly 95%, at a thousand replications each, read from 93.9% to 96.5% — and a table like that flags at least one of its correct cells on 69.9% of honest runs. Ten times the replications does not repair it: at ten thousand the same table still flags one 63.3% of the time.

method · Seeds

A block at every starting row

The t interval on the block count reads its scale from the disjoint blocks, three of them when 120 rows are cut into blocks of thirty-two, and throws away the other eighty-six blocks the series contains. Reading a block at every starting row leaves the scale's attenuation where it was — the two expectations agree to within a hundredth at every cell — and nearly doubles the degrees of freedom it realises, so the interval loses a third of its width at the same coverage. Where the interval falls short, at short blocks, it falls short by the same amount however the blocks are read.

taper · Bootstrap

A schedule that reads the mean

The block sizes may be anything at all provided they are functions of the contrasts. Two natural schedules break that, in opposite directions — and the most natural mistake of the three is not a schedule at all but a stopping rule, at 86.87% coverage and fewer observations.

pace · Stopping

A second break on a flat profile

Searching a hundred and twenty rows for one change point manufactures five units of likelihood. Searching for a second manufactures four more, on a series that has at most one — and on a profile whose whole range is under seven.

charged · Break point

The interval after a stop it chose

A rule that stops when the estimated precision is good enough stops on the samples whose estimate was small. Its interval covers 90% and claims 95%, and a fresh sample of the same random size covers 95.4%.

guarantee · Stopping

The optimum is a ratio, and its interval is sometimes the whole line

The best setting is −b₁/2b₂: a ratio of two estimates whose denominator is a curvature the design can often barely see. The delta method reports a finite interval every time and covers 68.8% where the curvature is weak; Fieller's set covers 95% and says so by being unbounded.

surface · Optimum

What a design chosen from the data costs

Two earlier measurements found what happens when a rule reads the data, and the error rate broke both times. A design that reads the data to decide where to put its runs breaks nothing — and the control that proves it also finds what the real shortfall is.

robust · Local design

What a reference distribution costs to sample

A randomisation test on a trial too large to enumerate has to sample its reference distribution, at 1/p attempts per draw and a p-value resolved to 1/(B + 1). Six constraints cost 9,878 attempts per thousand draws, and a thousand draws resolve p to 9.99·10⁻⁴ and not one digit finer.

product · Reference

What a two-arm rule may not pool

A spread computed "within the block" without the arm label carries a share of the effect, so the trial runs 173 observations at a null and 282 at an effect of 1.5. The stopping rule is reading the thing it exists to measure, and the phrase that produced it is one word long.

contrast · Allocation

What the interval covers

Eight rules and windows, and not one of them reaches its promised 95%. The range is 80.8% to 91.0%, and the choice between two block windows is a choice inside a shortfall that is four times larger.

readout · Bootstrap

The count or the length

A block length and a block count are one number read two ways at one sample size. Read at three, the studentised interval's width penalty tracks the count — with an R² of 0.9911 against a closed form that has no length in it — and its coverage tracks the length.

student · Bootstrap

The variance between imputations

Pooling several filled datasets covers 94.10% at two imputations and reaches its promise at five, where a single fill covered 85.78%. The correction everybody quotes is the smaller of the two doing the work — 1.00 ± 0.22 points against 1.55 ± 0.28.

missing · Missingness

Either model, but not neither

The augmented estimator's bias is −0.0085, −0.0083 and −0.0016 wherever one nuisance model is right, against components off by 0.8064 and 0.8190. One step past the overlap sweep it is the least biased estimator on the table at 0.0857 and the worst on it at 1.9265.

weights · Weighting

The same draws for both methods

Two intervals computed on the same simulated datasets give a difference in coverage whose variance can be 4.891 times smaller than on separate datasets — or, for a pair that covers different samples, 1.164 times larger. Which one a comparison gets is an exact sum over the counts each interval covers, and a standard error that ignores the sharing covers 100.00% for one pair and 93.07% for the other.

method · Seeds

The outcomes a trial could have stopped with

A trial that stops at its second look with z = 3.3 has a two-sided p-value of 0.000969, 0.000987, 0.00187 or 0.0421, depending on how the outcomes it could have stopped with are ordered. One of the four orderings does not change when the looks the trial never reached are replanned, and the same one gives a trial that ran to the end with z = 6 a p-value of 0.0256.

sequential · Stopping

An upper limit is the finding

A survey of a thousand people with a 90%-sensitive, 95%-specific test, estimating a prevalence of one in a thousand, prints a corrected 95% interval that covers the truth 95.17% of the time — and reaches below zero in 97.95% of surveys and lies entirely below zero in 3.35%. Coverage is not the problem. The problem is what the interval can say, and the honest summary of a typical survey is one number: the prevalence is below 1.64%. Bringing that limit down to twice the truth takes 184,521 people at this specificity and 5,083 with a test that never gives a false positive.

screening · Baserate

A length chosen for the interval

The squared-error rule for a block length weighs the scale's bias against its variance. Rebuilt for a block at every starting row, whose variance is two thirds as large, it asks for blocks fourteen per cent longer and buys four tenths of a point of coverage. A t interval has already paid for the scale's variance in its multiplier, so its own coverage is predicted by the bias alone — within seven tenths of a point up to blocks of twenty-eight rows — and a rule that chooses the shortest length that prediction says will cover asks for blocks two and a half to three times longer and lands within about a point of 95% at 120, 240 and 480 rows. The price is width, and it is paid where the series is short.

taper · Bootstrap

A break that is there

Put a real break in the regression at row 60 and the profile's two-unit set still misses it on most draws — it contains the true row on 12% of draws at a small break and 69% at a large one, without whitening; with the window searched as well, 23% to 75%. A set that would contain it on 95% of draws needs a threshold of about 19 units without whitening and about 8.5 with it, and either honest set holds most of the sample: fifty-seven of the seventy-three candidate rows at a break of one standard deviation. Searching the window does not move where the break is found.

twice · Break point

Two intervals for one return level

Two 95% intervals read off the same fits of the same records, against a level known in closed form. The symmetric one covers 80.3% at twenty-five blocks and reaches only 89.0% at two hundred — and 99.24% of its misses are the interval sitting entirely below the truth, which is not the endpoint anybody expects to fail.

extreme · Extremes

An imputation model the analysis does not contain

A model that fills the gaps without a covariate the analysis fits attenuates that covariate's coefficient by exactly the missing share, 0.4 to 0.26, and moves the one it did carry by exactly γρf, 0.6 to 0.642. The reverse case is supposed to inflate the interval, and at four strengths of the extra knowledge it does not.

missing · Missingness

Right for the wrong reason

A robust standard error costs no coverage where the risk is absent — 95.52% against 95.06% at twenty rows. It costs a 6.89% wider interval and a variance estimate 2.572 times as variable, and the pre-test that would avoid paying recovers 15.9% of what the insurance is worth.

sandwich · Misspecification

A simulation that stops when it looks settled

A simulation of an interval that covers exactly 95%, checked every 250 replications for a significant departure and stopped when it finds one, flags that correct interval on 29.54% of runs. Stopped instead as soon as its estimate reaches 95%, it reports an interval that covers 94% as meeting its level on 37.21% of runs. Stopped when the estimate stops moving, it reports the right number — and has quietly chosen to run about fifteen hundred replications.

method · Seeds

Where the derivative is zero

The delta method reads a standard error off a tangent line, and at a flat point the tangent says the spread is zero. The interval built on it for a squared mean covers 99.991% there and 85.978% one and a half standard errors away, with nearly every miss on the same side — and the law it should have used is a χ², not a normal.

normal · Clt

The interval with no resampling in it

Replace 1.96 in a normal interval on the block-means variance with Student's t on one fewer degrees of freedom than there are whole blocks, and resample nothing. Across twenty-four cells it covers at least as often as the studentised bootstrap interval at every one, by 0.42 to 10.42 points; it is narrower wherever seven blocks or fewer are left; and at fifteen blocks of 32 it covers 95.0%, which no resampled interval on the grid reaches.

student · Bootstrap

An interval for the winner

The largest of twenty correlated analyses, reported because it cleared the family-wise threshold, carries an ordinary 95% interval that covers its true mean 75.96% of the time at an effect of two standard errors and never at all when there is no effect. An interval built on the fact that it won covers 94.90% — and it is 2.48 times as wide, excludes zero for only 14.10% of the winners, and has no lower limit at all for 14.32% of them. It restores the promise by declining to say most of what the ordinary interval said.

paths · Forking

The fifth a third stage never reaches

A three-stage stopping rule keeps the two-stage rule's independence and the sequential rule's budget. With a first stage of five it covers 88.4% — worse than the rule it was meant to repair — because in a fifth of experiments the second stage buys nothing.

guarantee · Stopping

A share read off the sample

A block length chosen for an interval's coverage needs the share of the mean's variance the scale will find, and the published rule reads it from a first-order autoregression at the sample's lag-one value. Reading it from the sample's own autocovariances instead was supposed to survive series nobody had modelled. At 120 rows it covers 92.5% to 93.5% on all four laws — worse than the wrong shape on every one — because a centred series' autocovariances sum to exactly zero and the share they imply is too generous; read to lag forty it covers 84%. The assumed shape costs width, not coverage: 17% too wide where dependence stops at lag four, and nothing visible under a break, where the interval covers for a reason the share knows nothing about.

taper · Bootstrap

The interval the test inverts to

A randomisation test of no effect becomes an interval for the effect by testing every value at once: subtract the hypothesised effect from the treated outcomes and ask the same question again. Because the rule reads only covariates, one set of re-run allocations answers every value. Under minimisation the unadjusted t interval covers 99.7% at a width of 1.229; the inverted test of the same unadjusted difference covers 95.3% at 0.816, a third narrower, with no model of the outcome. Under complete randomisation the two are the same interval — which is what says the third was the rule's.

covadapt · Assignment

An interval that knows the band was maximised

A regression whose coefficients and eight-lag error band are maximised together prints an interval that covers 72% to 83% where 95% was promised, because it reports nearly the precision of a known covariance and delivers less. Two repairs were named. A sandwich standard error read off the whitened residuals repairs nothing — 71% to 83% — because the maximisation has already made those residuals look independent. A parametric bootstrap that redraws from the fitted band and maximises the band again on every redraw repairs about half the gap, to 85.5% to 89%, by pricing the maximisation through repeating it. What it cannot price is that the band it redraws from is the maximised one.

family · Dependence

A bootstrap that repeats the design

After a design that chose its settings from the data, a parametric bootstrap can keep the settings the experiment ran or repeat the whole procedure from the start. On twelve runs repeating it changes coverage by a point at most, because the adaptivity was nearly free. What matters is the interval's shape: the percentile bootstrap covers 90.6%, no better than the Wald interval's 91.3%; the studentised one 93.0%; the profile interval 94.9%. And after a design stuck at a twelvefold wrong guess, the studentised bootstrap — the textbook repair — covers 77.8%, with intervals averaging 54.5 wide for a parameter of 3.

robust · Local design

A ranking that belongs to the interval

Five ways of choosing a block length, read on the coverage of the interval each length builds rather than on the variance it implies. On the percentile interval the variance's verdict mostly survives: under a five-period moving average the plug-in and the whole-sequence rule tie at 120 rows and the sequence rule leads at 480. Build a studentised interval from the same resamples and the lead reverses — the plug-in's longer blocks cover 94.5% against 93.3% — because an interval that widens for its own uncertainty is paid for a long block where a percentile interval is punished for one. Under long memory no length and no interval covers two times in three.

feasible · Bootstrap

A set calibrated at the estimate

A confidence set for where a regression breaks needs a threshold on the profile likelihood, and no fixed one works: the honest threshold falls as the break grows. Simulating the profile's deficit at the true break, shift and persistence gives a set that covers 93.7% to 95.0% at every break size and narrows from 67 rows to 23. Simulating it at what one sample can estimate gives a set that covers 64% to 76%, and most of the shortfall is one number: a split regression's residuals report a persistence of 0.62 where the errors have 0.8.

twice · Break point

A shrink towards the answer

This line of essays reported that the 95% point is a dearer reading than a variance, that a further point wants a shorter block, and that a taper beats a rectangle on what a test reads where a variance says the opposite. All three came from one uncentred quantile. Read about the resamples' own mean, the point costs 28.5% against the variance's 45.8%, every level wants about the same block, and the windows order alike on all readings. The taper's coverage had a separate cause: its resampled mean is 0.875 of the sample's, and every series here had a true mean of exactly zero.

readout · Bootstrap

The mechanism the data cannot see

Two worlds produce identical covariates, identical patterns of what is recorded and identical recorded outcomes, to the last bit. Their true slopes are 0.6 and 0.315452, and the truth moves at 0.284548 per unit of an assumption nothing in the data can inform.

missing · Missingness

The count that is not the rows

Three hundred rows in five clusters of sixty carry 6.9000 times the variance an independent-rows calculation reports, and the interval that counts rows covers 53.42%. The same five unequal sizes laid out two ways give design effects of 9.3158 and 5.4652.

sandwich · Misspecification

A ratio whose interval has to be the whole line

The delta interval for a ratio of two means covers 95.61% when the denominator is eight standard errors from zero and 1.10% at a ten-thousandth of one, and ten times as wide it still covers only 3.48%. Linearising is not the fault. Gleser and Hwang proved that every interval that is always finite fails the same way, so an interval that keeps its promise has to be the whole line some of the time.

normal · Clt

The draws aimed at the tail

The chance a standard normal exceeds 5 is 2.8665×10⁻⁷, and a plain simulation needs 349 million draws to estimate it to within ten per cent. Draws aimed at the tail and weighted back need 565. Aimed slightly too narrowly, the same method has an infinite variance, an interval that covers 86.0% and gets worse with more draws, and an effective sample size that reads healthier than a proposal that works.

method · Seeds

An interval for the analyses admitted to

An analyst runs twenty correlated analyses, admits to one, and reports the winner with an interval conditioned on having cleared 1.96 — the selection as written down rather than as it happened. The interval covers 95.3% of the time, against 95.6% for one conditioned on all twenty, because a winner's truncation point is set by the threshold far more than by its unseen rivals: hiding nineteen analyses moves it by 0.061 on average. What the hiding changes is how often there is a winner at all. With no effect anywhere, a family of twenty produces a reported winner 35.5% of the time against 4.8% when all twenty are admitted, and one whose interval excludes zero 1.48% of the time against 0.13% — the interval keeps its promise and the report does not.

paths · Forking

The sums under a moving level

A trial's block sums are a free second estimate of its variance only while they see what the differences see, and a level that moves between blocks with an effect moving against it breaks that: pooled raw at a ceiling they cover 87.6% to 90.0%. Residualising the sums on an unbalanced design does not make them safe. It moves the failure. On ten and two, 90.0% at the ceiling becomes 93.8%, the trough leaves for a slope of about −1.5, and over the rest of the range the same estimate over-covers, at 99.9% on an interval half as wide again as it needs to be.

contrast · Blocking

An effect that is not one number

A randomisation test becomes an interval for the effect by testing every constant effect in turn, so when the effect grows with the prognosis every hypothesis it tests is false. It does not fall apart. It covers the population's mean effect within a point of its level at every slope measured, and its width grows by exactly what the sample's own composition adds. A regression that models the interaction stays narrow at 0.72, covers the sample's mean at 95%, and covers the population's at 84%. The extra width was never a defect. It was the difference between two questions.

covadapt · Assignment

Intervals for the findings

Benjamini–Hochberg's findings usually go out each with its ordinary 95% interval. With ten real effects of two standard errors among twenty tests, 11.59% of those intervals miss their effect, every miss on the far side, and the interval around the most prominent finding covers 72.36% of the time — 2.38% when the effects are one standard error. Intervals widened for the number of findings hold the share that miss under 5%.

multiplicity · Multiplicity

The assumption that identifies the mechanism

A selection model estimates how strongly an outcome decides whether it is recorded — the quantity two identical datasets showed no statistic can see — and it does so by assuming the outcome is normal. Where that holds and the outcome does decide, it repairs a slope complete cases put at 0.4318 to 0.5795. Where the missingness is at random and the residual is merely skewed, it reports selection that is not there, moves the slope from 0.5971 to 1.0319, and rejects missingness at random in 72.5% of studies.

missing · Missingness

An interval that covers and says nothing

A procedure returning the whole line 95% of the time and the empty set otherwise has coverage exactly 95% at every parameter value. Two real intervals at forty observations have expected widths of 0.2418 and 0.2417 and worst-case coverages of 55.31% and 92.21%.

intervals · Coverage

A flat point with more than one direction

At a stationary point of a function of several means the second-order law is ½ Z′HZ, so the bias is half the Hessian's trace — 2.008 for a bowl, 5.028 for a valley, and −0.006 for a saddle, where the eigenvalues cancel. The saddle's coverage is the worst of the three.

normal · Clt

A winner chosen by its diagnostic

An analyst runs twenty correlated analyses and reports the one whose diagnostics look cleanest, if it clears 1.96 — and a diagnostic is rarely independent of the statistic it checks. With no effect anywhere, a diagnostic correlated 0.9 with its statistic lets a winner through in 13.07% of families against 2.52% for an unrelated one. The interval conditioned on clearing 1.96 still covers 93.27%, but its misses have moved: it excludes zero in 4.69% of reported winners, where a 95% interval should do so in 2.5%. Conditioning on the diagnostic as well repairs it, and so does something a reader can do alone — weight the reported statistic by the chance an analysis like it wins the diagnostic contest, from three numbers a report can state.

paths · Forking

The cleanest of the significant

An analyst with twenty correlated analyses can choose in two orders: take the one whose diagnostics look cleanest and report it if it clears 1.96, or take the analyses that cleared 1.96 and report the cleanest of them. The second is the one described as prudent, because it never reports an analysis whose diagnostics look poor — and with no effect anywhere it reports a significant analysis in about 18% of families whatever the diagnostic is, against 2.55% to 13.15% for the first. The diagnostic decides which significant analysis is shown and never whether one is. Each report's own interval can still be made exact, by conditioning on a selection set that is a union of pieces rather than a half-line; the family's claim count cannot.

paths · Forking

A count that bets against its interval

A reader who sees only the count can bet against an interval's 95%, and win whatever the proportion is, exactly when some set of counts carries less than 95% at every proportion. Out of forty trials the textbook interval has one — a count of five or fewer, or thirty-five or more, covers at most 94.90% — and the score interval has none: a prior exists under which every one of its forty-one intervals is 96.46% credible, and that prior is the proof that no bet can be found.

intervals · Coverage

Testing for a flat point first

Test whether the gradient is zero, and if the test does not reject, read the interval off the second-order law that holds at a stationary point. For a squared mean that two-stage rule covers about 50% at its worst whatever the pretest's level — 49.72% with a 5% test, where the tangent interval it was meant to rescue never falls below 85.98% — because its worst sits exactly where the true distance equals the test's critical value, half the samples are declared flat there, and the second-order interval cannot reach the truth from any of them.

normal · Clt

Where the extremal index matters to a return level

A return level from a declustered fit depends on how often clusters arrive, which is the extremal index estimated from the same record. For a level reached once in two periods of a twenty-period record the index's error is a quarter to four fifths of the level's variance, and an interval that leaves it out covers 59.0% when clusters are long; put in, 90.0%. For a level reached once in a hundred periods its share is under two and a half per cent and adding it changes no interval at all — what fails there is the shape, fitted from however many clusters the dependence left: 82.3% coverage from forty-six clusters, 65.7% from nineteen.

extreme · Extremes

An interval that knows its finding was found

An interval for one Benjamini–Hochberg finding can be read from the distribution of its estimate given that it cleared the threshold it had to clear. It covers the finding's effect 94.8% of the time with real effects of two standard errors, against the ordinary interval's 87.0%, and it covers the most prominent finding 91.3% of the time where the ordinary one covers 72.4%. The price is width — a median 5.06 standard errors against 3.92 — and something larger: for three findings in four at that effect size the interval reaches back past zero, so a finding that cleared the line is, once the line is accounted for, not distinguishable from nothing.

multiplicity · Multiplicity

The interval the joint fit reports

Maximising a regression's coefficients and an eight-lag covariance band together gives a better estimate than plugging the band in. It also prints an interval that covers 71% to 85% of the time where 95% was promised — on the white predictor too, where every other rule is honest — because it reports almost the precision of the law's own covariance and delivers less than half of that advantage. Counting the eight lags it estimated buys one to two points.

timeseries · Dependence