Paired t-Test

In plain language

The paired t-test measures whether the average change within the same patients — between a before measurement and an after measurement — is large enough that it is unlikely to be due to chance alone. Instead of comparing two separate groups, it computes each patient's individual change (the difference), then asks whether the average of those changes is significantly different from zero. Removing between-patient variation by working with differences rather than raw values makes this test considerably more sensitive than comparing two independent groups of the same size. One important caveat: in a simple before-after claims study with no comparison group, a statistically significant change does not prove the change was caused by any intervention — it may simply reflect a background trend or a quirk of selecting patients with extreme baseline values.

What the paired t-test actually does

The paired t-test is deceptively simple in its mechanics but powerful in its logic. You have n subjects, each measured twice — at baseline and follow-up, or in a matched pair. Let d_i = X_i2 - X_i1 be the within-subject difference. The test is then a one-sample t-test on the d_i against the null hypothesis that the population mean difference mu_d = 0:

t = d_bar / (s_d / sqrt(n))

where d_bar is the sample mean of differences, s_d is the standard deviation of differences, and n is the number of pairs. The test statistic follows a t-distribution with n - 1 degrees of freedom under the null. The 95% confidence interval is:

d_bar ± t_{0.975, n-1} * (s_d / sqrt(n))

The key insight is that this is not a comparison of two separate groups; it is a one-sample test on the derived differences. This seemingly cosmetic reframing has a profound practical consequence for power.

Why pairing gains power: the variance algebra

If you were to ignore the pairing and run a two-sample Welch t-test on the pre and post values as if they were independent groups, the variance of the mean difference would be:

Var(X_bar_post - X_bar_pre) = sigma^2_post/n + sigma^2_pre/n

But for paired data, the variance of the mean difference is:

Var(d_bar) = Var((X_post - X_pre)_bar) = (sigma^2_post + sigma^2_pre - 2rhosigma_post*sigma_pre) / n

where rho is the within-subject (pre-post) correlation. When rho > 0 — which is almost always true in longitudinal clinical data, because patients who are high at baseline tend to remain relatively high at follow-up — the paired design subtracts a positive term from the variance. The higher the correlation, the greater the variance reduction, and the tighter the SE, and therefore the greater the power relative to an unpaired analysis of the same data. As a practical rule of thumb:

• rho >= 0.5: the paired design roughly halves the variance of the estimator
• rho >= 0.8: the paired design's SE is less than half that of the unpaired design.
• rho near 0: little gain from pairing; the paired test has fewer degrees of

This variance algebra is why lab value and biometric studies (where measurements are highly correlated within person over short periods) gain enormously from the paired design, while acute randomized crossover studies with long washout periods may see more modest gains.

The normality assumption: what must actually be normal

The paired t-test requires that the differences d_i are approximately normally distributed — not the original measurements, and not both groups jointly. In practice this is a weaker requirement than it may sound, for two reasons:

First, even if the original pre and post distributions are right-skewed (as is common for costs and utilization in healthcare), the differences often have a more symmetric distribution because the skew is partially shared across the two measurements on the same person.

Second, the Central Limit Theorem protects the t-statistic at larger n: for n >= 30, departures from normality in the differences rarely invalidate the test for inference on the mean difference. At n < 15 with strongly non-normal differences, the Wilcoxon signed-rank test is the preferred alternative (see relations below).

As with all t-tests, resist the temptation to run a Shapiro-Wilk test on the differences to "confirm normality" before choosing the test. At small n, Shapiro-Wilk has low power to detect non-normality; at large n, it will flag trivial departures that the CLT renders irrelevant. Inspect the differences with a histogram and Q-Q plot instead, and use subject-matter knowledge about the outcome distribution.

RWE application: pre-post cost and utilization analyses

The paired t-test is the workhorse for within-patient pre-post comparisons in claims and EHR analyses. Common contexts:

• PPPM cost pre vs post a diagnosis or intervention: identify an index event

• Lab values before and after drug initiation in EHR: HbA1c, LDL, eGFR, and

• Utilization rates in matched cohort studies: when PS-matched pairs are formed

The RWE traps: regression to the mean and secular trends

The paired t-test is technically correct only for within-person comparisons; it cannot remove bias from the study design. Two traps recur in applied work:

Trap 1 — Regression to the mean

Suppose you select a cohort of high-cost claimants — say, patients in the top decile of total annual spending — and then compare their costs in the next year using a paired t-test. You will almost certainly observe a statistically significant reduction in mean costs. But this reduction is partly or entirely a statistical artifact, not a true effect. Patients who are selected because they had an extreme value in one period will, on average, have a less extreme value in the next period even if nothing has changed about them. This is regression to the mean. The paired t-test faithfully measures the difference, but the difference has nothing to do with any intervention. The fix is a concurrent control group (unexposed patients in the same period) or a difference-in-differences design that subtracts the secular trend from both the treated and control group.

Trap 2 — Secular trends masquerade as treatment effects

A pre-post analysis with no concurrent control group cannot distinguish between the effect of an intervention and background trends that would have occurred anyway: seasonal variation in respiratory admissions, a secular decline in hospitalizations due to policy changes, a regression-to-the-mean artifact (see above), or maturation of a disease cohort. All of these confound a naive pre-post paired t-test. The paired t-test is appropriate as the estimator conditional on the study design being adequate to identify the causal effect — which a simple pre-post design usually is not. Interrupted time series, difference-in-differences, or a controlled pre-post with a matched unexposed group are the design-level fixes; the paired t-test remains the appropriate within-pair estimator once those design features are in place.

Trap 3 — The matched cohort analysis debate

In propensity-score-matched cohort studies, patients are matched one-to-one on observed confounders. The conventional question is: should the primary analysis use the paired t-test (treating matched pairs as dependent observations) or an unpaired analysis on the matched sample? The strict statistical position is that if you matched on covariates, the outcomes of matched pairs are dependent and a paired analysis is correct. In practice, both robust variance estimators (which account for within-cluster dependence without explicitly pairing) and paired analyses are seen in the HEOR literature, and simulation evidence suggests that correctly specified robust variance estimates and paired analyses give similar results. The conservative and defensible approach is to use the paired t-test (or the paired Wilcoxon signed-rank test as a sensitivity check) when matched pairs are clearly defined one-to-one.

Pros, cons, and trade-offs

Pros: - Removes between-subject variability from the error term, yielding substantially narrower confidence intervals and greater power than an unpaired two-sample test when the pre-post correlation is moderate to high (rho >= 0.4). - Produces a directly interpretable effect estimate: the mean within-person difference (e.g., mean reduction of $340 per patient per month) with a CI on the original scale. - Robust to moderate non-normality of differences at n >= 30 (CLT). - Straightforward CI formula; extends naturally to regression with a fixed subject effect when covariates need adjustment. - The simplest correct choice for paired continuous data; avoids the complexity of mixed models when only two time points are present.

Cons: - Requires complete pairs: a patient missing either the pre or post measurement must be excluded or imputed. This can introduce selection bias if missingness is not random (patients lost to follow-up often differ systematically from completers). - Cannot handle more than two time points; for three or more repeated measurements, linear mixed models (LMM) or MMRM are the appropriate generalization. - Blind to secular trends and regression to the mean — it faithfully estimates the mean within-person difference, but that difference may not be the causal effect of interest in an observational pre-post design. - At small n (< 15) with non-normal differences, the Wilcoxon signed-rank test is more reliable. - The mean difference is the target quantity; if the distribution of differences is extremely right-skewed (e.g., cost differences in patients with rare high-cost events), the mean is an unstable estimator and a gamma GLM with a subject random effect or a bootstrap CI may be preferable.

When to use

• Two time points, continuous outcome, same subjects at both times: the paired
• One-to-one matched case-control or cohort studies with a continuous outcome: treat
• Crossover trials with two treatment periods: within-subject comparison of the two
• When n >= 15 and the differences are approximately symmetric: the normality
• When a mean difference is the clinically or economically meaningful estimand: for

When NOT to use

• Independent groups: do not apply the paired t-test to unpaired data. Using a
• More than two time points: three or more repeated measurements require linear
• Skewed differences at small n: if n < 15 and the histogram of differences is
• *Pre-post design without a concurrent control when secular trend or regression to
• Binary or ordinal outcomes: for paired binary outcomes (e.g., readmission yes/no

Interpreting the output

In the worked example, six patients enrolled in an ED diversion care management program averaged 2.0 fewer emergency department visits in the 12 months after enrollment compared with the 12 months before (mean difference d̄ = 2.0). The standard deviation of within- patient differences is 0.894, the standard error is 0.365, the t statistic is 5.48 on df = 5, the two-sided p-value is approximately 0.003, and the 95% CI on the mean paired difference is [1.06, 2.94] visits.

(1) Formal interpretation. The observed mean within-patient reduction of 2.0 visits is 5.48 standard errors above zero. Under the null hypothesis that the population mean of within-patient differences is zero, a t statistic this large on df = 5 arises by chance in approximately 0.3% of samples. The 95% CI [1.06, 2.94] is constructed so that if the study were repeated many times, approximately 95% of such intervals would contain the true mean paired difference — it is consistent with a true reduction of between about 1 and 3 visits per patient.

(2) Practical interpretation. All six patients reduced their visits after enrollment and the average reduction of 2.0 visits is statistically distinguishable from zero. However, this is a pre-post design without a concurrent control group, and these patients were enrolled specifically because of high baseline utilization. Some or all of the reduction may reflect regression to the mean rather than a program effect. A difference-in-differences design comparing this cohort to a matched unexposed group over the same period would be needed to attribute the reduction causally to the care management program.

Worked example

Scenario

A health plan enrolled six high-utilizer patients in an emergency-department (ED) diversion care management program. An analyst wants to know whether the mean number of ED visits per patient changed after enrollment, using a paired t-test on the pre-program (12 months before) and post-program (12 months after) visit counts. The six patients had ED visit counts of 5, 3, 6, 4, 7, and 3 in the pre-period, and 3, 2, 3, 2, 4, and 2 in the post-period. The analyst computes within-patient differences, then manually walks through the paired t-test arithmetic.

Dataset

ED visits in the 12 months before and 12 months after enrollment in a care management program. The "difference" column is pre minus post for each patient.

Steps

• Step 1 — Compute the mean difference. Sum the six within-patient differences: 2 + 1 + 3 + 2 + 3 + 1 = 12. Divide by n = 6: mean difference d_bar = 12/6 = 2.0 visits. Every patient reduced their ED visits; the average reduction is 2 visits.

• Step 2 — Compute the variance of the differences. For each difference, subtract d_bar = 2.0 and square the result: (2-2)^2=0, (1-2)^2=1, (3-2)^2=1, (2-2)^2=0, (3-2)^2=1, (1-2)^2=1. Sum of squared deviations = 0+1+1+0+1+1 = 4. Sample variance s_d^2 = 4/(n-1) = 4/5 = 0.8.

• Step 3 — Compute the standard deviation and standard error. Standard deviation of differences: s_d = sqrt(0.8) = 0.894. Standard error: SE = s_d / sqrt(n) = 0.894 / sqrt(6) = 0.894 / 2.449 = 0.365.

• Step 4 — Compute the t-statistic. t = d_bar / SE = 2.0 / 0.365 = 5.48. Degrees of freedom = n - 1 = 6 - 1 = 5.

• Step 5 — Determine the p-value and confidence interval. With t = 5.48 and df = 5, the two-sided p-value is approximately 0.003, well below alpha = 0.05. The 95% CI uses the critical value t_{0.975, 5} = 2.571: lower = 2.0 - 2.571 0.365 = 2.0 - 0.94 = 1.06; upper = 2.0 + 2.571 0.365 = 2.0 + 0.94 = 2.94. The 95% CI is approximately [1.1, 2.9] visits.

• Step 6 — Interpret with caution. The mean reduction of 2.0 ED visits per patient (95% CI: 1.1 to 2.9) is statistically significant. However, this is a pre-post design without a concurrent control group. Patients were enrolled because they had very high baseline utilization; regression to the mean would predict some decline even without the program. A difference-in-differences design comparing this cohort to a matched unexposed group in the same period would be needed to attribute the reduction causally to the care management program.

Result

Mean difference (pre minus post) = 12/6 = 2.0 ED visits. Sum of squared deviations = 4.0; s_d^2 = 4/5 = 0.8; s_d = 0.894; SE = 0.894/2.449 = 0.365. t = 2.0/0.365 = 5.48, df = 5, p approximately 0.003 (two-sided). 95% CI: 2.0 +/- 2.571*0.365 = [1.06, 2.94] visits. Conclusion: statistically significant average reduction of 2 ED visits per patient, but a concurrent control group is needed to rule out regression to the mean before attributing the reduction to the program.

Runnable example

import math
from scipy import stats

# ── Dataset: ED visits pre and post care management enrollment ──
# Six patients; pre-period and post-period 12-month visit counts.
pre  = [5, 3, 6, 4, 7, 3]
post = [3, 2, 3, 2, 4, 2]

# ── Step 1: compute within-patient differences (pre minus post) ──
diffs = [p - q for p, q in zip(pre, post)]   # [2, 1, 3, 2, 3, 1]
n = len(diffs)

# ── Step 2: mean difference ──
d_bar = sum(diffs) / n                        # 12/6 = 2.0
print(f"Differences:    {diffs}")
print(f"Sum of diffs:   {sum(diffs)}  (n={n})")
print(f"Mean diff (d̄): {d_bar:.4f}")

# ── Step 3: variance and SD of differences ──
ss = sum((d - d_bar)**2 for d in diffs)       # sum of squared deviations = 4.0
s2 = ss / (n - 1)                             # sample variance = 4/5 = 0.8
s  = math.sqrt(s2)                            # SD of diffs = sqrt(0.8) = 0.8944
se = s / math.sqrt(n)                         # SE = 0.8944/sqrt(6) = 0.3651
print(f"\nSum sq deviations: {ss:.4f}")
print(f"Variance (s²):     {s2:.4f}")
print(f"Std dev (s):       {s:.4f}")
print(f"Std error (SE):    {se:.4f}")

# ── Step 4: t-statistic and two-sided p-value (manual) ──
t_manual = d_bar / se
df = n - 1
p_manual = stats.t.sf(abs(t_manual), df=df) * 2
print(f"\nt-statistic (manual): {t_manual:.4f}")
print(f"Degrees of freedom:   {df}")
print(f"p-value (two-sided):  {p_manual:.4f}")

# ── Step 5: 95% confidence interval ──
t_crit = stats.t.ppf(0.975, df=df)           # 2.5706 for df=5
ci_lo = d_bar - t_crit * se
ci_hi = d_bar + t_crit * se
print(f"t-critical (0.975):   {t_crit:.4f}")
print(f"95% CI:               [{ci_lo:.2f}, {ci_hi:.2f}] visits")

# ── Step 6: confirm using scipy.stats.ttest_rel ──
# ttest_rel computes the paired t-test directly on the original arrays.
t_scipy, p_scipy = stats.ttest_rel(pre, post)
print(f"\nscipy.stats.ttest_rel:  t={t_scipy:.4f}, p={p_scipy:.4f}")
print("(Identical to manual calculation — confirms the implementation.)")

# ── Interpretation note ──
print(f"\nMean reduction: {d_bar:.1f} ED visits per patient (95% CI: {ci_lo:.1f}, {ci_hi:.1f})")
print("WARNING: No concurrent control — regression to the mean cannot be ruled out.")
print("         A difference-in-differences design would be needed for causal inference.")

# ── Dataset: ED visits pre and post care management enrollment ──
pre  <- c(5, 3, 6, 4, 7, 3)
post <- c(3, 2, 3, 2, 4, 2)
n    <- length(pre)

# ── Step 1: within-patient differences ──
diffs <- pre - post
cat("Differences:", diffs, "\n")
cat("Sum:", sum(diffs), "  Mean:", mean(diffs), "\n")

# ── Step 2: variance of differences (manual) ──
ss <- sum((diffs - mean(diffs))^2)
s2 <- ss / (n - 1)
s  <- sqrt(s2)
se <- s / sqrt(n)
cat(sprintf("Variance: %.4f  SD: %.4f  SE: %.4f\n", s2, s, se))
cat(sprintf("t = %.4f  df = %d\n", mean(diffs)/se, n - 1))

# ── Step 3: paired t-test via t.test ──
# paired = TRUE: t.test internally computes pre - post and runs a one-sample test.
res_paired <- t.test(pre, post, paired = TRUE, conf.level = 0.95)
cat("\n--- t.test(paired = TRUE) ---\n")
print(res_paired)

# ── Step 4: equivalence — one-sample t-test on differences ──
res_one <- t.test(diffs, mu = 0, conf.level = 0.95)
cat("\n--- t.test(diffs, mu=0) — identical result ---\n")
print(res_one)

# ── Step 5: effect size (Cohen's d_z) ──
# For paired t-test, Cohen's d_z = mean(diffs) / sd(diffs)
dz <- mean(diffs) / sd(diffs)
cat(sprintf("\nCohen's d_z (effect size for paired design): %.4f\n", dz))
cat("Interpretation: d_z >= 0.8 is a large effect by convention.\n")

# ── Wilcoxon signed-rank as a sensitivity check ──
wsr <- wilcox.test(pre, post, paired = TRUE, exact = FALSE)
cat(sprintf("\nWilcoxon signed-rank (sensitivity): W=%.1f, p=%.4f\n",
            wsr$statistic, wsr$p.value))
cat("If p-values from paired t-test and Wilcoxon are consistent, the normality\n")
cat("assumption is not load-bearing for this dataset.\n")