Normal Distribution
nothing is called 95% until it has been counted
Concept

Upper confidence limit — where it appears

The upper end of a one-sided confidence interval, read as a value the quantity exceeds only at the stated rate. On skewed data or with an estimated spread, the end of a symmetric two-sided interval is exceeded more often than that.

Named by 7 essays across 3 fields — each of them below, with the objects they name alongside it.

The power trials actually have when sized for 80% from a pilot of 10. Four thousand pilots of 10 observations, each sizing a trial for 80% power at half a standard deviation from its own standard deviation. 55.9% of the trials have less than 80% power and 11.1% less than 50%; the median trial has 76.8%.

The spread a pilot supplies

A trial sized for 80% power from a pilot's standard deviation is sized from an estimate that is too small more often than not. With a pilot of ten, 55.9% of the trials it sizes have less than 80% power and 11.1% less than 50%, although the planned sample is right on average. Sizing from the pilot's 80% upper confidence limit instead leaves 19.8% short, at 1.65 times the sample; from its 90% limit, 10.0% short at 2.12 times.

planned · Power Four 95% intervals for the mean of 30 exponential observations, split by the side they miss on. Both bars should read 2.5%. The t interval misses below the mean on 6.38% of samples. Widened until its total is exactly 5%, it misses below on 4.69% and above on 0.30%. Hall's transformation misses below on 3.31% and above on 1.96%.

The side a bound is read from

On thirty exponential observations the upper limit of a 95% t interval is exceeded by the true mean 6.38% of the time, against the 2.5% a safety margin set from it assumes. Widen the interval until its total coverage is exactly 95% and the upper limit is still exceeded 4.69% of the time. A symmetric repair fixes the number that is reported and not the one that is used; Hall's transformation, which bends the interval, takes the same rate to 3.31%.

tails · Student Five upper limits for the mean of 15 observations from a exponential source, by margin and by failure rate. t limit: exceeded 8.51% at a mean margin of 0.523; Hall's transformation: exceeded 4.61% at a mean margin of 1.038; gamma family, shape known: exceeded 2.59% at a mean margin of 0.787; gamma family, shape by likelihood: exceeded 4.04% at a mean margin of 0.746; gamma family, shape by skewness: exceeded 17.51% at a mean margin of 0.421. The maximum-likelihood shape has a median of 1.10 and the skewness-based shape 3.20, against a true shape of 1.

A limit from the family it came from

On fifteen exponential observations an upper limit for the mean built from the gamma family's own exact pivot, with the shape estimated by maximum likelihood, is exceeded 4.04% of the time at a mean margin of 0.746 standard deviations. Hall's transformation is exceeded 4.61% at 1.038, and the same family limit with the shape read from the sample skewness is exceeded 17.51%. Knowing the family and estimating its one parameter well recovers most of what knowing the population buys — and the parameter is estimated well only from the logarithms.

tails · Student Expected loss of three upper limits against the level each is set at, fifteen exponential observations, penalty 20. A failure costs 20 standard deviations of margin. At the conventional 97.5% the t limit's loss is 2.143, Hall's 1.922 and the fitted family's 1.530. Each at its own best level: the t limit 1.263 at 99.95%, Hall's 1.862 at 98.50%, the family 1.308 at 99.25%. The best fixed multiple of the standard error costs 1.255.

The level a limit should be set at

When a failure costs twenty times a standard deviation of margin, a t limit on fifteen exponential observations set at the conventional 97.5% has an expected loss of 2.143. Set at 99.95%, the same formula's loss is 1.263 — within 1% of the best any fixed multiple of s/√n can do, and ahead of both Hall's transformation and the fitted gamma family at their own best levels. Hall's is the worst at every level on every source. Choosing the level does more than choosing the construction, and the correction that reads the sample is the one no level rescues.

tails · Student What a corrected critical value delivers to a t statistic: actual size over nominal, upper tail, 30 exponential draws. At a nominal 1% the upper-tail t test's actual size is ×0.153 with Student's quantile, ×2.94 with one Cornish–Fisher term and ×1.51 with Hall's cubic, both using the source's skewness, and ×0.878 with Hall's cubic using each sample's own. At 10⁻⁶ they are under a hundredth of nominal, ×1.04e+4, ×50.0 and ×1.59.

The correction a t test would use

On thirty exponential draws the one skewness term that held a sum's critical value within 10% out to five standard deviations nearly triples a t test's 1% size when the source's skewness is known, and its upper critical value turns back — at a level of 0.89% on ten draws, where a test asked for one in a thousand rejects 121 times too often. Hall's monotone cubic removes the turn and gains a floor. Built from the sample's own skewness it holds the short side within 20% from 10% down to one in a hundred thousand, and leaves the long side, the one a safety limit reads, at twice its level: the samples that fail are the ones that hide their skewness.

expansion · Rate Expected loss of three upper limits against the level each is set at, when a failure is charged by its size, fifteen exponential observations, penalty 20. A failure costs 20 times the distance, in standard deviations, by which the mean exceeds the limit. At the conventional 97.5% the t limit's loss is 0.728, Hall's 1.136 and the fitted family's 0.835. Each at its own best level: the t limit 0.726 at 97.00%, Hall's 0.805 at 90.00%, the family 0.735 at 92.50%.

A failure charged by its size

Charge an upper limit's failure by how far the true mean exceeds it rather than by a fixed amount, and the level a limit should be set at falls further than any other change in this comparison moves it: on fifteen exponential observations, a penalty of twenty per standard deviation of shortfall asks for a t limit at 97% where a penalty of twenty per failure asked for 99.95%. The failures are small whichever limit produces them — about an eighth of a standard deviation — so a charge by size is a small charge per failure. The ranking hardly moves: the t limit stays within about 1% of the fitted family, whose failures are no smaller than its own, and Hall's transformation stays last by 11%.

tails · Student What it costs to set the t limit at one level without knowing which skewed source the 15 observations came from. For the t limit on 15 observations, charged twenty per standard deviation of shortfall plus its margin, the loss at each nominal level as an excess over the best level for each source. The best levels are 97% for the exponential, 96.25% for the gamma, shape 2, 97.75% for the gamma, shape ½, 98.25% for the lognormal. The minimax level, 97.25%, costs at most 0.8% on any of them: 0.1% on the exponential, 0.6% on the gamma, shape 2, 0.2% on the gamma, shape ½, 0.8% on the lognormal. The conventional 97.5% costs up to 1.0%.

A level set before the source is known

Every best level for an upper limit on skewed data was found knowing the source, and the fifteen observations that would tell the sources apart are the ones being used to set the limit. Chosen instead to protect the worst of four skewed sources, the t limit's level is 97.25% when a failure is charged by the size of its shortfall, and it costs at most 0.8% more than each source's own best — the sources disagree about the best level and agree that missing it a little is cheap. The fitted gamma family, the construction meant to adapt to the source, is the one that needs it: its best levels run from 90% to 96.5% and its minimax level costs 3.4%. Charged a fixed amount per failure, the t limit's minimax level is 99.97% and costs 6.5%; the conventional 97.5% costs up to 119.9%.

tails · Student

Named alongside it

The objects these essays reach for when they reach for this one.

SkewnessStudent's tSample sizeCoverageDecision theoryEstimated varianceExpected lossSignificance levelExpected shortfallChi squaredConditional coverageCornish–Fisher expansion

All concepts