The t statistic wearing different clothes

A regression output prints a slope, a standard error, a t statistic, a p-value and an $R^2$. It looks like five numbers about the fit. Two of them are the same number.

$$t^2 \;=\; \frac{(n-2)\,R^2}{1 - R^2}$$

exactly, for a simple regression, on every dataset. Not approximately, not in large samples: the identity holds algebraically, and recomputing t from $R^2$ and the sample size over five hundred fitted datasets reproduces the fitted t to a worst relative disagreement of 8.4 × 10⁻¹⁶, which is the floating-point arithmetic and nothing else.

What follows #

Given n, $R^2$ and t are in one-to-one correspondence, and so are $R^2$ and the p-value. So:

$R^2$ carries no information the p-value does not. A reader who has the p-value and the sample size can compute $R^2$ and vice versa. The two are printed side by side as though they were a measure of strength and a measure of evidence, and they are one quantity in two parameterisations.

$R^2$ cannot be a measure of practical importance and statistical significance at the same time, which is how it is usually deployed: “the effect was significant (p = 0.003) and explained 18% of the variance”. That sentence contains one fact.

The distinction between statistical and practical significance is real and $R^2$ is not where it lives. The standard advice — do not confuse a small p-value with a large effect — is correct, and the number offered as the antidote is a monotone function of the p-value.

The number that is not a function of the p-value is the one a sweep across designs identifies: the residual spread in the outcome’s units, and the slope in the outcome’s units per unit of predictor. Those are the quantities a reader converts into a decision, and neither is recoverable from $R^2$ and n.

Two studies with the same R-squared and the same precision #

The identity has a consequence that is easier to see than to believe.

The left study’s points are scattered 0.50 about the line and the right study’s 2.00. Both have an $R^2$ of 0.85, both have a slope of 1, and — at the same sample size — both have the same standard error for the slope, to every digit: 0.0664.

That last agreement is not a coincidence of the construction, it is the identity again. $R^2$ fixes t given n, and t is the slope over its standard error, so with the slope fixed the standard error is fixed too. Two studies with the same $R^2$, the same n and the same slope have the same precision necessarily.

What they do not have is the same relationship to the data. One predicts a new observation to within ±1 and the other to within ±4, and no number in either regression output distinguishes them except the residual spread — which is usually printed, usually as “residual standard error”, and usually not read.

That is the same failure four datasets with one summary demonstrates, with one difference worth noting: there the four datasets were constructed by hand and look contrived, and here the two are produced by a formula and both look entirely ordinary.

The construction is not a search. Given a target $R^2$ and a residual spread, the design that produces it is a closed form — $\mathrm{var}(x) = \sigma^2 R^2/(\beta^2(1 - R^2))$ — so the pair exists for every $R^2$ between 0 and 1 and every ratio of spreads whatever. There is no residual spread so large that a design cannot rescue its $R^2$, and none so small that a design cannot ruin it.

Reading the identity backwards: what R-squared is worth at a given n #

The correspondence works in both directions, and reading it the other way gives a practical table nobody prints.

At ten observations, an $R^2$ below 0.399 is not significant at the conventional level. At two thousand, an $R^2$ of 0.002 is — and it is significant at p = 0.05 while explaining a fifth of one per cent of the outcome’s variance.

Both halves of that are usually stated as warnings and neither is usually stated as arithmetic. “A large sample makes trivial effects significant” and “a small sample cannot detect anything” are the same identity read at its two ends, and the table converts them into numbers a reader can check against the paper in front of them.

The operational use is short. Given a paper’s n, the $R^2$ at which its p-value would be exactly 0.05 is a fixed number, and comparing the reported $R^2$ with it says immediately whether the finding is comfortably clear of the threshold or sitting on it. That is the same information the p-value gives, which is the point of this page — but it is information the reader can get from $R^2$ alone when the p-value has been omitted, which happens more often than the reverse.

Why the identity is not better known #

It is derivable in three lines and it is in every regression textbook as an exercise, so the question is why the two numbers are read as independent.

They are computed by different routines. $R^2$ comes from an analysis-of-variance decomposition and t from a coefficient’s standard error, and the two arrive in different parts of the printed output with different names. Nothing in the presentation says they are a reparameterisation.

The identity is specific to simple regression. With several predictors, the overall $R^2$ relates to the F statistic for the whole model rather than to any coefficient’s t, and individual coefficients have their own t statistics that $R^2$ does not determine. So a reader who has met the multiple-regression case has met a situation where the two genuinely differ, and the simple case looks like a special case rather than the general one.

“Variance explained” sounds like a property of the relationship. The phrase is doing the damage. A share of variance is a share of the outcome’s variance in this sample, and the sample’s variance is partly the design, which is what a sweep across designs shows.

The multiple-regression version, and where it stops #

For a model with k predictors the overall statistic is

$$F \;=\; \frac{R^2/k}{(1-R^2)/(n-k-1)},$$

which is the same relationship with k in it: given n and k, $R^2$ and F determine each other exactly. So the overall $R^2$ is still the overall p-value in different clothes, and the same argument applies to the model as a whole.

What changes is the individual coefficients. Each has a t that depends on that predictor’s own partial variance after the others are accounted for, and the overall $R^2$ does not determine any of them. A model with $R^2$ = 0.9 can have every coefficient insignificant — if the predictors are collinear the fitted surface is well determined and its decomposition into coefficients is not — and a model with $R^2$ = 0.1 can have one strongly significant coefficient among many null ones, which is the family-wise problem in a regression’s clothing.

So in the multiple case $R^2$ and significance genuinely come apart, at the level of individual coefficients and not at the level of the model. That is a narrower escape than it sounds, because the sentence “the model explained 18% of the variance and the effect was significant” is about the model and the effect, and the first half of it is still the model’s own p-value.

The same identity for a correlation, and why it matters more there #

A correlation coefficient between two variables is the square root of the $R^2$ of the regression of either on the other, so everything above transfers unchanged — and in the setting where correlations are most used it does more damage.

A correlation of 0.30 in a sample of a hundred has a t of 3.11 and a p-value of 0.0024. A correlation of 0.30 in a sample of twenty has a t of 1.33 and a p-value of 0.198. The same number described as a “moderate correlation” in both papers is strong evidence in one and no evidence in the other, and the convention of describing correlations by size bands — weak, moderate, strong — attaches a verbal label to a quantity whose meaning depends on a number the label omits.

The bands are also design-dependent in exactly the way a sweep across designs measures: the same underlying relationship measured on a restricted range reports a “weak” correlation, and measured on a wide one reports a “strong” one.

So a reported correlation carries two contaminations at once. Its evidential weight depends on n, which the label omits, and its size depends on the spread of the sample, which the label also omits. Neither is a defect in the coefficient — it is exactly what it is defined to be — and both are defects in the practice of quoting it alone.

The repair is the same three numbers as below, with the addition of the two variables’ standard deviations, from which the regression slope in real units can be recovered. A correlation without those is a standardised quantity whose standardisation nobody has stated.

Where the identity does buy something #

Nothing on this page says $R^2$ is useless, and there is one thing the identity makes it good for that its usual reading does not.

Converting between studies’ reported statistics. A meta-analysis frequently has papers reporting different things: one gives a t statistic, one an F, one an $R^2$, one only a p-value and a sample size. The identity turns any of those into any other, exactly, for a simple regression — so a pooled analysis can use every study rather than the subset that happened to report the same summary. That is a real and unglamorous use, and it works precisely because the quantities carry the same information.

Checking a paper’s internal consistency. If a reported $R^2$, n and p-value do not satisfy the identity, one of the three is wrong. That is a cheap audit with no false positives — the relationship is exact, so a mismatch beyond rounding is an error rather than a judgement — and it is the same device as recomputing a test statistic from a reported mean and standard error, which finds transcription errors in published tables at a rate nobody expects.

Both uses treat $R^2$ as a redundant encoding of the same fact, which is what it is. Neither treats it as a measure of anything, which is what it is not.

There is a third and it is the one $R^2$ was invented for. Before a computer printed standard errors as a matter of course, the analysis-of-variance decomposition was the practical route to a test, and $R^2$ was the quantity that decomposition produced. Its survival into outputs that also print the standard error is an artefact of that history rather than a judgement that both are wanted, which is worth knowing because it explains why a number with so little to recommend it is in every table.

Two routes, and the refusal #

The identity is checked on fitted data rather than derived on paper, which is the distinction this site keeps.

The fitted route runs an ordinary least-squares fit on five hundred simulated datasets and reads t off the coefficient and its standard error, as any software would.

The reconstructed route takes that fit’s $R^2$ and the sample size, and computes $\sqrt{(n - 2)R^2/(1 - R^2)}$. It never touches the standard error.

The worst relative disagreement across five hundred datasets is 8.4 × 10⁻¹⁶. That is not agreement within tolerance; it is the same number computed twice, and the check is written with a tolerance of 10⁻¹⁰ so that an implementation which had, for instance, divided by n instead of n − 2 would fail rather than pass as an approximation.

The refusal is the reading this page exists to refuse — $R^2$ taken as saying something about how far the points are from the line. The check requires two studies with the same $R^2$ to have the same residual spread, which is what a reader assumes when they compare two $R^2$ values as if they compared scatter, and it must fail: the pair is constructed with spreads a factor of four apart and an $R^2$ equal to twelve decimal places.

What to print instead, in three numbers #

The regression output already contains everything a reader needs, and the reading order is the thing that is missing.

The slope, in the outcome’s units per unit of predictor. This is the finding. It answers what happens if the predictor moves.

The residual standard error, in the outcome’s units. This is the model’s usefulness. It answers how close a prediction will be, and it is the width of a prediction interval divided by about two.

The slope’s standard error, or the interval around the slope. This is the evidence, and it subsumes the t statistic, the p-value and $R^2$, all three of which are functions of it given the slope and the sample size.

$R^2$ is then a derived quantity that can be computed if anyone wants it and carries nothing that is not already present. A paper reporting those three and omitting $R^2$ has lost no information; a paper reporting $R^2$ and omitting the residual standard error has lost the only number that was about the data rather than about the arithmetic.

What a reader should do with the three numbers #

The reading order above is a prescription, and it is worth walking one output through it, because the order changes which questions get asked.

Start with the slope. A model of systolic blood pressure on daily sodium gives, say, 0.9 mm Hg per gram. That is the finding, it is in units a clinician can hold, and it can be compared with the effect of any other intervention on the same outcome.

Then the residual standard error. If it is 16 mm Hg, the model places an individual’s pressure within about ±32 mm Hg, which is most of the clinical range, and the model is useless for predicting an individual whatever its $R^2$. If it is 4, it is not.

Then the interval on the slope. If it runs from 0.3 to 1.5, the finding is that sodium matters by somewhere between a little and a moderate amount; if it runs from −0.1 to 1.9, the study has not established the direction.

A fourth step is worth adding when the model will be used rather than read. Ask what the predictor’s range was in the sample, because the slope is only established over that range and the residual spread is only measured there. A slope of 0.9 mm Hg per gram estimated over intakes of 2 to 4 grams says nothing about 10 grams, and the regression output contains no warning that the question is outside the data.

Nowhere in those steps is $R^2$ needed, and at no point would knowing it have changed a judgement. What it would have changed is the impression: an $R^2$ of 0.03 reads as a negligible relationship and an $R^2$ of 0.4 as a strong one, and the same slope and residual spread can produce either, depending on how variable the sodium intake was in the people sampled.

The same discipline applied to a p-value is the fleet’s standing complaint about single numbers, and this is the sharper version of it: the p-value at least answers a question, and $R^2$ answers the same one in a way that sounds like a different one.

Still open: whether the standardised coefficient is any better #

The usual alternative to $R^2$ for comparability is the standardised coefficient — the slope in standard deviations of y per standard deviation of x — and it has the same defect in a form that is harder to see.

β·s_x/s_y contains the sample’s own s_x, so it is a design-dependent quantity by the same argument as $R^2$. In fact for a simple regression it is the correlation coefficient, so it is $R^2$ again under the square root, and everything on this page applies to it unchanged.

With several predictors it is not the correlation any more, and whether the design contamination survives in the same form is not obvious: the standardisation divides by each predictor’s own spread, which cancels some of the effect, and the outcome’s spread in the denominator still carries all of them. Whether the cancellation is partial or complete, and whether a standardised coefficient is therefore comparable across studies in a way $R^2$ is not, is a question with a closed-form answer that nothing here has worked out.