A leak read off a two-by-two table

A cut walked into the tail reduced a whole regression to two probabilities. A rule that balances a threshold on each of two covariates leaves some share of their interaction — the centred product of the two threshold indicators — unbalanced, and that share depends on the dependence between the covariates only through one number: $p_{11}$, the chance that a unit is past the cut on both. Given the share $p$ past each cut, the quadrant formula turns $p_{11}$ into the leak in closed form, and the copula, the correlation and the shape of the dependence everywhere else in the plane drop out.

That essay ended on the practical consequence and declined to measure it. A trial planning to balance on a clinical threshold — a dose, a laboratory value that defines a condition — could estimate $p_{11}$ from a historical cohort and read the leak off the formula before a single unit was randomised. The prediction it wrote down was specific: at the 90th percentile the estimate would be sharp enough to separate the copulas, at the 99th it would not, and the 99th is exactly where the copulas’ leaks differ most.

The prediction turns out to be right in direction and wrong about the mechanism. The estimate at the 99th is not merely noisy. On most cohorts of realistic size it is a fixed number, the same number on every copula, and that number is one hundred per cent.

What a cohort hands the formula #

Nothing about a cohort enters the calculation except its two-by-two table at the cut: how many units are past both thresholds, past one only, past the other only, and past neither. The cut is a fixed value on each covariate — the clinical threshold — so the two margins are estimated along with $p_{11}$, and the plug-in leak is the formula fed the three proportions the table supplies. Because nothing else enters, the cohorts here are drawn as tables directly, multinomial on each copula’s exact four cell probabilities, rather than as pairs of covariates that are then counted. The two are the same law and the first is cheaper by the cohort’s size.

The setting is the one the tail essay swept: two standard normal covariates, five copulas each matched at a Spearman rank correlation of 0.4 — the Gaussian, the t on four degrees of freedom, the Frank, the Clayton with its dependence in the lower tail and the same Clayton turned over — and cuts placed at the 90th and 99th percentiles of each covariate. Six hundred cohorts are drawn at each of three sizes, 300, 1,000 and 3,000 units.

At the 90th percentile the plug-in does what a plug-in should. Its median across cohorts of a thousand is within half a point of the true leak on all five copulas — 19.2% against 19.2% under the Gaussian, 29.0% against 28.9% under the t, 43.6% against 43.1% under the turned-over Clayton — and the middle eighty per cent of cohorts land in a band about fourteen points wide. That is a usable estimate. A trial told that its threshold rule will leave between thirteen and twenty-six per cent of the interaction alone knows something it did not know before, and knows it from a table rather than from a fitted model of the dependence.

At the 99th percentile the picture is not a wider version of the same thing. Under the Frank, whose true leak is 2.2%, the middle of the cohorts’ estimates runs from 14.4% to 100%, and the median is 100%. Under the Clayton, true leak 0.6%, the same. Under the Gaussian, true leak 13.7%, the median is 34.2% and the upper decile is the full hundred. The estimates are not centred on the truth with a wide spread around it. They are piled against the top of the scale, and the pile is tallest exactly on the copulas whose true leak is smallest.

Three cells, three coefficients #

The pile has an exact cause, and it is arithmetic rather than sampling noise. The leak is the share of the centred product $ab$ that a regression on the two indicators removes, where $a = A - p$ and $b = B - q$. A two-by-two table has four cells, so in the cohort the product takes at most four values, one per cell. The regression has three coefficients: an intercept and one on each indicator. If any cell is empty, the product takes at most three values on three patterns of $(A, B)$, and three coefficients fit three points exactly. The residual is zero, the share removed is one, and the leak is one hundred per cent — on every copula, at every true $p_{11}$, whatever the cohort’s other three counts are.

The empty cell that matters at a far cut is the one past both thresholds. With nobody past both, $AB$ is zero on every unit, the product is $pq - qA - pB$, and that is a straight line in the two indicators. A cohort that has seen no unit past both cuts has not measured a large leak. It has measured nothing, and the formula, asked for a share, returns the one share the arithmetic allows.

Every one of the thirty settings sits on one curve. A full leak happens exactly when a cell is empty — on all thirty, with no exception — and, out where it matters, the empty cell is the joint one, so the share of cohorts reading a full leak is close to the chance a Poisson count with mean $np_{11}$ is zero. That mean is the only quantity on the horizontal axis, and it is small in a way the size of a cohort does not suggest. At the 99th percentile the Gaussian puts 0.093% of units past both cuts, so a cohort of a thousand expects 0.93 of them and 40.5% of such cohorts have none. The Frank expects 0.27 and 75.0% have none. The Clayton, whose lower-tail dependence leaves its upper tail nearly independent, expects 0.17, and 82.7% have none.

At three hundred units the 99th percentile is worse in a second way. One cohort in ten has no unit at all past the cut on one of the covariates, the margin reads zero, and the formula has nothing to divide by; those cohorts return no estimate, and are left out of every count that follows. Of the cohorts that do return one, 78% of the Gaussian’s and 95% of the Clayton’s read a full leak.

At the 90th percentile the same curve explains why nothing goes wrong. The smallest expected joint count on the slate is 4.9, under the Clayton at three hundred units, and at that mean an empty corner happens to fewer than one cohort in a hundred. The 90th percentile is not safe because its leaks are smaller or its copulas better behaved. It is safe because a cohort of a few hundred has several units in its corner.

The bootstrap agrees with the empty cell #

The interval a practitioner would attach to the plug-in is the percentile bootstrap: redraw the cohort with replacement, recompute the leak, read off the 2.5th and 97.5th percentiles of the redrawn values. On a table that is three hundred redrawn tables, multinomial on the cohort’s own cell proportions, and it costs nothing.

At the 90th percentile it holds. Coverage runs from 93.2% to 98.3% across the fifteen combinations of copula and size, which is a percentile bootstrap doing its job on a smooth function of three proportions. The interval narrows as the square root of the cohort: between thirty-four and forty points wide at 300 units, thirteen and twenty-two at 1,000, eight and thirteen at 3,000.

At the 99th it fails, and the failure is the empty cell again. A cohort with no unit past both cuts has a redrawing law that puts zero probability on that cell, so every redrawn table also has it empty, every redrawn leak is one hundred per cent, and the interval is the single point $[100\%, 100\%]$. The bootstrap does not report that it knows nothing. It reports, with an interval of zero width, that the rule will leave the whole interaction alone. Coverage at three hundred units is 0.6% under the Gaussian and 0.0% under the Frank and the Clayton, and the median interval width on those three copulas is zero.

Where the copula does put units in the corner, the same bootstrap recovers. The turned-over Clayton — the one copula with its dependence in the upper tail, where the cut is — expects 4.1 units past both at a thousand and the interval covers 94.7%; the t, with dependence in both tails, covers 88.3% at a thousand and 97.3% at three thousand. Under the Gaussian it takes three thousand units to reach 92.3%. Under the Frank and the Clayton no size drawn here is enough: 33.0% and 0.8% at three thousand. The interval is trustworthy exactly on the copulas that make the cut’s corner populated, which is the information the cohort was supposed to supply.

This is the hole near zero that no sample size fills in a sharper form. Wilson’s interval for a proportion loses coverage at a small expected count and keeps it from collapsing; a resampling interval built on a count of zero cannot leave zero, because resampling a zero gives zero. The redrawing has no way to put probability on a cell the sample left empty, and the quadrant formula has no way to read an empty cell as anything but a perfect fit.

An interval that holds by inversion #

The repair the empty cell points to is to stop resampling the count and bound it instead. A Clopper–Pearson interval for $p_{11}$ — the exact binomial interval on the count past both — is well defined at a count of zero, where its lower end is zero and its upper end is close to the familiar rule of three — $3.7/n$ for the two-sided interval. Carrying every value of $p_{11}$ in that interval through the formula at the cohort’s own margins, and taking the smallest and largest leak that results, gives an interval for the leak by inversion. The leak is not monotone in $p_{11}$ — it is one at $p_{11} = 0$, falls to exactly zero at independence and rises after it — so the range is read off the whole interval, not off its ends.

It holds. Coverage is at least 98.5% at every one of the fifteen settings on the 99th percentile, and at least 97.3% on the 90th — conservative, as a Clopper–Pearson interval is, and never short. What it reports at the far cut is the honest version of the bootstrap’s false certainty. At three hundred units the median interval runs from zero, or within a point of it, to above ninety-eight per cent on every copula: the cohort cannot say whether the rule leaves none of the interaction or all of it. At a thousand units it still runs from zero to above ninety-six per cent on the Gaussian, the Frank and the Clayton. At three thousand the Gaussian’s median interval is 0.5% to 43.0% and the Frank’s zero to 40.1%, which is an upper bound worth having — the rule leaves less than half — and no more than that.

Only the two copulas with dependence in the upper tail get an interval whose lower end is clear of zero by more than a point, and they need a thousand units to do it: 2.9% to 97.3% under the t and 17.6% to 99.0% under the turned-over Clayton. By three thousand those are 15.8% to 67.0% and 33.4% to 84.1%. For the copulas without upper-tail dependence the cohort’s corner stays nearly empty at every size drawn, and the interval says so by refusing to close.

That is the advice the tail essay guessed at, with a number attached. At a cut in the far tail, a cohort supports a bound on the leak and not an estimate of it. The bound is useful when it is low enough to act on, which at the 99th percentile needs a cohort in the thousands, and it is useless at the few hundred units a planning dataset usually has. The quantity an upper limit is the finding argued for in a prevalence survey is the same quantity here, for the same reason: when the count that carries the information can be zero, the interval’s job is to say how large the thing might be, and a point estimate is a number the data did not produce.

What a thousand units can tell apart #

The tail essay’s motive for wanting the estimate was discrimination. The five copulas agree on their rank correlation and disagree on the leak, and a cohort that could say which leak it had would be saying something about the dependence a rank correlation cannot. The bootstrap interval at the 90th percentile is where that question can be asked, since it is where the interval holds.

A cohort of a thousand separates the copulas whose leaks are far apart and mostly fails to separate the ones that are close. Drawn from the Gaussian, its interval rules out the turned-over Clayton’s 43.1% on 99.3% of cohorts and the Clayton’s 4.4% on 82.2%; it rules out the t’s 28.9% on 41.3% and the Frank’s 12.2% on 24.3%. The three radially symmetric copulas — the Gaussian, the t and the Frank — are told apart from each other between a quarter and four fifths of the time, and the pairs that sit within ten points of each other are told apart less often than not. The diagonal is each interval’s own miss rate, between 2% and 7%.

That ordering is the honest reading of how much information a two-by-two table carries about a copula. The table is four numbers and its leak is one function of three of them; two copulas with similar diagonals at the cut give similar tables, whatever they do elsewhere, and no amount of arithmetic on the table can distinguish them. The Gaussian and the Frank differ by seven points at the 90th percentile and by eleven at the 99th, and at the 90th a thousand units tells them apart one time in four. At the 99th, where they differ most, the cohort has nothing in its corner and tells nothing apart.

A median split on one side #

The tail essay left a second calculation undone, and it is short. A common design cuts only one covariate at a clinical threshold and splits the other at its median, because only the first has a cut anyone would write down. The median split has an exact zero of its own — a centred median split’s square is a quarter identically — and the obvious hope is that the zero carries over: one factor at the median, the product protected.

The quadrant formula generalises to unequal shares $p$ and $q$ past the two cuts. The two covariances the regression needs are $(1 - 2p)c$ and $(1 - 2q)c$, with $c = p_{11} - pq$, and the leak is zero only when the vector of the two is zero — which needs both factors to vanish, or the indicators to be independent. A median split sets $1 - 2q$ to zero. It removes the second covariate’s own route into the leak and leaves the first one’s untouched.

The mixed cut does not merely fail to protect the product; far out it leaks more than two thresholds do on all five copulas — the t is the one exception nearer the middle, below them at the 90th — and on the Gaussian it reverses the tail essay’s result. Two thresholds on a Gaussian copula leak 19.49% near the 95th percentile and fall to 5.86% by the 99.9th, because the Gaussian has no tail dependence and two rare events stop co-occurring. A threshold beside a median split leaks 39.54% at the 95th and 76.53% at the 99.9th, and it is still climbing.

The reason is that the median split never becomes rare. As the threshold runs out, the product’s information about the interaction comes from how often a unit past the threshold is above the other covariate’s median, and for a positively related pair that chance does not fade. With $r$ that conditional chance at the edge of the first covariate’s range, the formula’s limit is $(2r - 1)^2$. Under the Gaussian, $r$ tends to one at any positive correlation, so the limit is the whole interaction. Under the turned-over Clayton it is one too, and the mixed cut reaches 98.13% at the 99.9th. The Frank’s conditional law at the edge gives $r = e^{\theta/2}/(1 + e^{\theta/2})$, a limit of 32.9%, and it is at 32.79%; the Clayton’s upper tail gives $r = 1 - 2^{-(1 + \theta)}$, a limit of 16.7%, and it is at 16.68%. The t’s limit has a closed form too, $T_{\nu + 1}\bigl(\rho\sqrt{(\nu + 1)/(1 - \rho^2)}\bigr)$ for $r$, and sits at 43.9%.

So the property that decides a two-threshold rule’s far tail — tail dependence — is not the one that decides a mixed rule’s. A mixed rule’s limit is set by how far the body of one covariate’s law moves when the other is extreme, and a Gaussian copula, which has no tail dependence at all, moves it as far as it can go. The design that cuts one covariate because only one has a clinical cut is the one that leaves the most of their interaction unbalanced.

Two routes and a refusal #

Every share on the mixed sweep is computed twice on each copula’s grid: by the generalised closed form fed the grid’s own cell probabilities, and by a direct Gram solve of the product on an intercept and the two indicators, weighted by the grid. The two agree to $3 \times 10^{-14}$ at all fifty-five cuts. At equal cuts the generalised formula reduces to the tail essay’s to $10^{-12}$, and its limit $(2r - 1)^2$ is checked on the formula itself at a cut of one in a billion.

The cohort sweep found one defect in the formula before it found anything in the data. A table with both off-diagonal cells empty — four units past both cuts and none past exactly one — makes the two indicators identical, the regression singular, and the determinant a rounding error above zero; the formula then returned 88.9% for a product that is $(A - p)^2$ and fitted exactly by one coefficient. It now reads such a table as the full leak it is. Below the Fréchet bounds, where no law has the three proportions supplied, it returns nothing, which is how an earlier draft’s upper bound of 456% was caught.

The refusal attached to the section above is the sentence the median split’s zero invites: that a median split on one covariate protects its product with a threshold on the other. Checked at the 90th and 95th percentiles on every copula, it fails on all five.

Still open: a cut chosen for the cohort #

The measurement this points to is a design choice rather than an estimate. A trial free to move its threshold a little could choose the cut at which its cohort’s interval is narrowest, or at which the bound is lowest — the 95th rather than the 99th, say — and balance there. That buys an estimate and changes the interaction being protected, and whether the trade is worth it depends on how fast the true leak moves with the cut against how fast the cohort’s information falls. Both are computable on the five copulas here, and the comparison has not been made. A cut chosen by looking at the cohort would also be a selection, and its interval would have to be priced as one.

A second open question is the mixed cut’s own estimate. Its limit depends on $r$, which is a conditional chance at a moderate event — above the median — and so is far better estimated by a cohort than $p_{11}$ at a far threshold. The prediction is that the mixed cut’s leak is estimable from a few hundred units at cuts where the two-threshold leak is not, which would make it the one tail rule whose leak a trial can know in advance; it has not been run.

The claim, and where it stops #

This essay takes the field’s quadrant formula as an estimate from a cohort’s two-by-two table. The claims are that at a cut on the 90th percentile a thousand units put the plug-in’s median within half a point of the true leak on all five copulas, with the middle eighty per cent of cohorts in a band about fourteen points wide, and the percentile bootstrap covers between 93.2% and 98.3% at every size from 300; that a cohort’s plug-in leak is exactly one hundred per cent when and only when a cell of its table is empty, on all thirty settings; that at the 99th percentile that is 40.5% to 82.7% of cohorts of a thousand on the three copulas without upper-tail dependence, and the bootstrap interval collapses on them to a single point and covers between 0.0% and 43.2%; that an interval by inversion of a Clopper–Pearson interval covers at least 97.3% at every setting and at the 99th spans zero to one hundred per cent at three hundred units; that at a thousand units and the 90th percentile a cohort rules out a copula whose leak differs by more than twenty points on 98–100% of draws and the three radially symmetric copulas from each other on 24–85%; and that a median split on one covariate removes nothing of the leak a threshold on the other carries, and far out leaks more than two thresholds on all five, tending to $(2r - 1)^2$ — the whole interaction under a Gaussian.

What stays out, named as a decision: real cohorts, whose dependence near a clinical cut none of the five copulas need resemble; other rank correlations, at which every count above moves; a cut chosen from the cohort, deferred above; and an estimate of the copula from the cohort’s body carried out to the cut, which is a model where this essay uses only a table — the route the extremes field takes past the last observation, and one with assumptions of its own.

The boundary against the cut walked into the tail is that it computed the leak from each copula’s exact quadrant probabilities, and this essay asks what is left of the calculation when those probabilities have to be counted. Against the share a sample removes, which measured how much of an interaction a balancing rule removes within one trial’s sample, this essay is about the planning number before any trial exists. And against the interval that covers and says nothing, the inverted interval here is the same shape of honesty — coverage bought by width — arrived at because the count that would narrow it is zero.