Normal Distribution
nothing is called 95% until it has been counted
Concept

Factorial design — where it appears

Varying every factor at once so that each run informs every effect, rather than moving one factor at a time. It is more efficient at the same number of runs and it is the only arrangement that can see an interaction at all.

Named by 9 essays across 3 fields — each of them below, with the objects they name alongside it.

Where a D-optimal design puts its runs. The D-optimal measure over 121 candidate settings on a square region. It keeps 9 of them and discards the rest, and the 9 it keeps are the settings a catalogue would have offered without any of this arithmetic. What the search adds is the weights: 0.1458, 0.0962, 0.0802, which nine equal runs cannot express.

A design is a number

A standard design is taken from a catalogue and then measured. Turn the arithmetic round and a design becomes the answer to an optimisation — and over 121 candidate settings the search keeps nine of them, which are exactly the nine a catalogue would have offered, at weights nine equal runs cannot express.

optimality · Criterion Four runs, and the term they cannot reach. Every run sits at a corner, so x₁² and x₂² are 1 at every run and both columns are copies of the intercept. The normal matrix is singular: the design has no information about curvature at all, and no analysis can recover it.

The design that cannot see a curve

A two-level factorial has every run at a corner, where every squared term equals one — so the column that would estimate curvature is a copy of the intercept, and the design has no information about it at all. A few runs at the centre buy one number back, and only one.

surface · Factorial Twenty walks up the same hill, σ = 2. Each walk fits a plane to the same four-corner factorial, takes its gradient as a direction, and steps along it until a run comes in below the one before. The true optimum is the cross. 80% of the walks stop before the best point on their own path — not because the direction was wrong, but because one noisy run is enough to stop them, and the direction error costs only 3.9% of the available gain.

Walking up the gradient

The fitted gradient is wrong by an angle with a closed form, σ/(|β|√N), and what that angle costs is its squared cosine — twelve per cent at twenty degrees. What costs a third of the gain is not the direction at all. It is deciding where to stop.

surface · Optimum Two factors that interact — where each design looks. The true response at the four corners is -10, 6, 4, 0. One factor at a time visits three of them, sees that raising either factor alone helps, and recommends raising both — a corner it never ran, and one that is worse than either single change. It picks the best corner 0.0% of the time against the factorial design's 99.4%, on the same number of runs.

One factor at a time

Changing one thing per experiment estimates each effect from two conditions; changing everything at once estimates each from every run. The ratio is (k+1)/2 and it is exact — and when two factors interact, the one-at-a-time design recommends a setting it never tried.

design · Factorial A central composite design, 13 runs. Adding 4 axial runs at ±√2 gives every factor three levels, which is the least that can estimate a squared term. The normal matrix now inverts, so each βᵢᵢ has an estimate of its own — and at exactly this axial distance the design is rotatable, which the next figure measures.

Three levels, and the ring where the design says the same thing

A central composite design puts its axial runs at ±α, and α is not a matter of taste. At F to the quarter the prediction variance depends only on how far a point is from the centre and not at all on which direction it lies in — a property with no simulation in it, exact or absent.

surface · Factorial Adding a run can make the design worse. The D-efficiency of the best N-run design at each size, against the optimal measure. It is not a rising curve. 13 runs reaches 99.77% and 14 falls to 99.44%, because the optimal weights are real numbers and N runs is an integer approximation to them, so how good a design can be depends on how well N divides. A Wald interval behaves the same way: a larger sample sometimes makes its coverage worse, for exactly this reason.

The design that has to be integers

The optimal design is a set of real weights and an experiment is a set of runs, so the theory's answer is never available. Thirteen runs reach 99.77% of it and fourteen reach 99.44% — adding a run makes the design worse per run, and the search that finds it does not always find the same one.

optimality · Criterion What a 8-run fraction of 4 factors confounds. The defining relation is I = ABCD, so the resolution is 4. A is estimated as A + BCD; B is estimated as B + ACD; C is estimated as C + ABD; D is estimated as D + ABC. Each of those is an identity about the design rather than an approximation about the data.

The word a fraction costs

A half fraction estimates each main effect as an exact sum of that effect and everything it is confounded with — no error term, no sample-size argument. With every interaction at 0.8 the design reports a true effect of −1 as −0.20, and the design cannot test the assumption that makes the number mean anything.

design · Factorial The Box–Behnken design in three factors: 15 runs, none at a corner. Twelve runs at the midpoints of the cube's edges and 3 at its centre. Every run holds one factor at zero, so no run puts all three factors at an extreme — which is what makes it runnable where a corner is not. The three panels are the design's coordinate projections, with repeated positions marked.

The design that refuses the corners

Box–Behnken runs three factors in fifteen runs and puts none of them at a corner, which is what makes it usable where a corner cannot be run. It predicts the corner 1.84 times worse than the seventeen-run design that goes there, and 1.31 times worse at the middle of a face, and all three numbers are matrix computations with no simulation in them.

design · Factorial What one lost run costs a 16-run factorial fitting 11 coefficients. Every run is worth the same: dropping any one multiplies every coefficient's variance by 1.2000, which is 1 + 1/(N − p) with N = 16 and p = 11, and gives every pair of coefficients a correlation of 0.1667 where the complete design had exactly zero.

The run that did not happen

Lose one run from any orthogonal design and every coefficient's variance is multiplied by exactly 1 + 1/(N − p), and every pair of coefficients acquires a correlation of exactly 1/(N − p + 1) where there was none. The price is set by the design's spare capacity and by nothing else, and a saturated design cannot survive it at all.

design · Factorial

Named alongside it

The objects these essays reach for when they reach for this one.

Experimental designCentral composite designCurvatureLeast squaresPrediction varianceCentre pointStudy designBox behnken designD-optimalityDegrees of freedomDesign measureFractional factorial

All concepts