Normal Distribution
nothing is called 95% until it has been counted
Field

A covariance with no parameter

A regression's coefficients and its errors' dependence can be fitted together when the dependence is one number. When it is an estimated covariance there is nothing for “jointly” to mean — until a family is named, and then the family's own width is the parameter. Three things the measurement says, two of them the opposite of the guess: the plug-in's shortfall is the taper's rather than the data's and is nothing under two of four laws; nothing in the likelihood chooses the width, because nested families buy about a unit of it a lag and that is what a criterion charges; and a fit-only objective does not run away, because a unit diagonal fixes the trace.
Where the general fit becomes the parametric one. The band family's objective at the autoregression's own geometric sequence, cut off at each width, on one sample of 60 rows. The horizontal line is the profile likelihood the parametric fit maximises, written independently through a different whitening. At the full width the two are the same number to 3e-14, which is what says the general construction contains the parametric one rather than resembling it. Below 10 lags there is no line at all: the geometric sequence cut off short is not a covariance matrix, so the objective has nothing to evaluate. Between the two the truncation is briefly above the parametric likelihood — a wrong covariance can fit one sample better than the right one, which is the whole reason a width has to be charged for rather than chosen.

A family before a fit

A regression's coefficients and one correlation can be maximised together. Replace the correlation with an estimated covariance and there is nothing left for "jointly" to mean — until a set of covariances is named, and the set turns out not to contain the truth.

 Most of the rise is the optimiser's, and under one law it is not. The rise in log-likelihood from the tapered plug-in to the maximum over the same eight-lag band, beside what the same optimiser produces on a sample generated from the plug-in's own covariance — where the family is correctly specified by construction and there is nothing to find. Under AR(1) at 0.8 the raw rise is 5.72 and the manufactured baseline is 4.79, leaving 0.93 at 1.8 standard errors; under long memory the excess is 0.14, at 0.2. Under the moving average it is 11.87 at 19.4 standard errors, on every draw. The taper is a shrinkage, and it costs nothing where the sequence decays smoothly and a great deal where it stops dead.

The plug-in and the maximum

A tapered covariance estimate sits five and a half log-likelihood units below the maximum of the likelihood it is substituted into. Four fifths of that is what the optimiser would have found if nothing were missing.

 A wider band is always a better fit. The likelihood maximised over the band, at five widths, averaged over 30 samples. A band at L lags is a band at L + 1 with the last entry held at zero, so the families are nested and the maximised likelihood cannot fall — it does not, on any draw. What it does is rise at 0.984 of log-likelihood a lag. A parameter that is doing nothing buys half a unit in expectation and Akaike's criterion charges one, so this is a criterion very nearly indifferent between every width on offer. The dashed line is what a charge of one unit a lag would exactly cancel. Nothing in the fit chooses a width, and what does choose one is a charge somebody has to pick.

Nothing in the fit picks the width

A wider band is always a better fit, and it is better by about one unit of log-likelihood a lag — which is the order of what a criterion charges for a parameter. Three defensible rules choose widths a factor of three apart.

 Fitting them together is worth something under one law. Four fits of the same regression under four dependences: least squares, the two-step plug-in every whitened rule in this collection runs, the coefficients and the band maximised together, and a whitening at the law's own covariance that nobody has. Under the moving average — the one law the band family contains — the joint fit beats the two-step by 0.0077 at 3.3 paired standard errors. Under the autoregression, long memory and the break it is a tie: 0.4, 1.0, 0.3 standard errors. That is the same ordering the likelihood gap gave, arrived at through the coefficients rather than through the objective.

What fitting them together buys

Maximising over the coefficients and the covariance together beats the two-step under one of four dependences and ties under the other three. It is the one the band family contains, and the likelihood said so before any coefficient was compared.

 Six intervals for the white predictor, a five-period moving average. Under a five-period moving average, over 200 regressions of 120 rows with an eight-lag band, how often each 95% interval for the white predictor's coefficient covers it: two-step band, as reported 95.5%; joint fit, as reported 72.0%; joint fit, White's sandwich 71.0%; joint fit, four-lag sandwich 71.0%; joint fit, band maximised again 85.5%; the law's own covariance 92.0%. The bootstrap redraws each regression 24 times from its fitted coefficients and fitted band, maximising the band afresh on every redraw.

An interval that knows the band was maximised

A regression whose coefficients and eight-lag error band are maximised together prints an interval that covers 72% to 83% where 95% was promised, because it reports nearly the precision of a known covariance and delivers less. Two repairs were named. A sandwich standard error read off the whitened residuals repairs nothing — 71% to 83% — because the maximisation has already made those residuals look independent. A parametric bootstrap that redraws from the fitted band and maximises the band again on every redraw repairs about half the gap, to 85.5% to 89%, by pricing the maximisation through repeating it. What it cannot price is that the band it redraws from is the maximised one.

All essays