Paired t-Test

What the paired t-test actually does

The paired t-test is deceptively simple in its mechanics but powerful in its logic. You have n subjects, each measured twice — at baseline and follow-up, or in a matched pair. Let d_i = X_i2 - X_i1 be the within-subject difference. The test is then a one-sample t-test on the d_i against the null hypothesis that the population mean difference mu_d = 0:

t = d_bar / (s_d / sqrt(n))

where d_bar is the sample mean of differences, s_d is the standard deviation of differences, and n is the number of pairs. The test statistic follows a t-distribution with n - 1 degrees of freedom under the null. The 95% confidence interval is:

d_bar ± t_{0.975, n-1} * (s_d / sqrt(n))

The key insight is that this is not a comparison of two separate groups; it is a one-sample test on the derived differences. This seemingly cosmetic reframing has a profound practical consequence for power.

Why pairing gains power: the variance algebra

If you were to ignore the pairing and run a two-sample Welch t-test on the pre and post values as if they were independent groups, the variance of the mean difference would be:

Var(X_bar_post - X_bar_pre) = sigma^2_post/n + sigma^2_pre/n

But for paired data, the variance of the mean difference is:

Var(d_bar) = Var((X_post - X_pre)_bar) = (sigma^2_post + sigma^2_pre - 2*rho*sigma_post*sigma_pre) / n

where rho is the within-subject (pre-post) correlation. When rho > 0 — which is almost always true in longitudinal clinical data, because patients who are high at baseline tend to remain relatively high at follow-up — the paired design subtracts a positive term from the variance. The higher the correlation, the greater the variance reduction, and the tighter the SE, and therefore the greater the power relative to an unpaired analysis of the same data. As a practical rule of thumb:

• rho >= 0.5: the paired design roughly halves the variance of the estimator relative to an independent-samples design.
• rho >= 0.8: the paired design's SE is less than half that of the unpaired design.
• rho near 0: little gain from pairing; the paired test has fewer degrees of freedom (n-1 vs 2n-2) and can actually be slightly less powerful than the unpaired test.

This variance algebra is why lab value and biometric studies (where measurements are highly correlated within person over short periods) gain enormously from the paired design, while acute randomized crossover studies with long washout periods may see more modest gains.

The normality assumption: what must actually be normal

The paired t-test requires that the differences d_i are approximately normally distributed — not the original measurements, and not both groups jointly. In practice this is a weaker requirement than it may sound, for two reasons:

First, even if the original pre and post distributions are right-skewed (as is common for costs and utilization in healthcare), the differences often have a more symmetric distribution because the skew is partially shared across the two measurements on the same person.

Second, the Central Limit Theorem protects the t-statistic at larger n: for n >= 30, departures from normality in the differences rarely invalidate the test for inference on the mean difference. At n < 15 with strongly non-normal differences, the Wilcoxon signed-rank test is the preferred alternative (see relations below).

As with all t-tests, resist the temptation to run a Shapiro-Wilk test on the differences to "confirm normality" before choosing the test. At small n, Shapiro-Wilk has low power to detect non-normality; at large n, it will flag trivial departures that the CLT renders irrelevant. Inspect the differences with a histogram and Q-Q plot instead, and use subject-matter knowledge about the outcome distribution.

RWE application: pre-post cost and utilization analyses

The paired t-test is the workhorse for within-patient pre-post comparisons in claims and EHR analyses. Common contexts:

• PPPM cost pre vs post a diagnosis or intervention: identify an index event (new diagnosis, hospital admission, drug initiation), define a pre-period window (e.g., 6 or 12 months before index) and a post-period window (e.g., 6 or 12 months after), compute per-patient-per-month costs in each window, and test whether the mean difference is nonzero.

• Lab values before and after drug initiation in EHR: HbA1c, LDL, eGFR, and similar biomarkers measured at clinic visits naturally produce a pre-treatment and post-treatment measurement for each patient who has both visits captured.

• Utilization rates in matched cohort studies: when PS-matched pairs are formed one-to-one, the matched design creates a natural pairing — but see below for the field debate on whether paired analysis is mandatory.

The RWE traps: regression to the mean and secular trends

The paired t-test is technically correct only for within-person comparisons; it cannot remove bias from the study design. Two traps recur in applied work:

Trap 1 — Regression to the mean

Suppose you select a cohort of high-cost claimants — say, patients in the top decile of total annual spending — and then compare their costs in the next year using a paired t-test. You will almost certainly observe a statistically significant reduction in mean costs. But this reduction is partly or entirely a statistical artifact, not a true effect. Patients who are selected because they had an extreme value in one period will, on average, have a less extreme value in the next period even if nothing has changed about them. This is regression to the mean. The paired t-test faithfully measures the difference, but the difference has nothing to do with any intervention. The fix is a concurrent control group (unexposed patients in the same period) or a difference-in-differences design that subtracts the secular trend from both the treated and control group.

Trap 2 — Secular trends masquerade as treatment effects

A pre-post analysis with no concurrent control group cannot distinguish between the effect of an intervention and background trends that would have occurred anyway: seasonal variation in respiratory admissions, a secular decline in hospitalizations due to policy changes, a regression-to-the-mean artifact (see above), or maturation of a disease cohort. All of these confound a naive pre-post paired t-test. The paired t-test is appropriate as the estimator conditional on the study design being adequate to identify the causal effect — which a simple pre-post design usually is not. Interrupted time series, difference-in-differences, or a controlled pre-post with a matched unexposed group are the design-level fixes; the paired t-test remains the appropriate within-pair estimator once those design features are in place.

Trap 3 — The matched cohort analysis debate

In propensity-score-matched cohort studies, patients are matched one-to-one on observed confounders. The conventional question is: should the primary analysis use the paired t-test (treating matched pairs as dependent observations) or an unpaired analysis on the matched sample? The strict statistical position is that if you matched on covariates, the outcomes of matched pairs are dependent and a paired analysis is correct. In practice, both robust variance estimators (which account for within-cluster dependence without explicitly pairing) and paired analyses are seen in the HEOR literature, and simulation evidence suggests that correctly specified robust variance estimates and paired analyses give similar results. The conservative and defensible approach is to use the paired t-test (or the paired Wilcoxon signed-rank test as a sensitivity check) when matched pairs are clearly defined one-to-one.

Pros, cons, and trade-offs

Pros:

• Removes between-subject variability from the error term, yielding substantially narrower confidence intervals and greater power than an unpaired two-sample test when the pre-post correlation is moderate to high (rho >= 0.4).
• Produces a directly interpretable effect estimate: the mean within-person difference (e.g., mean reduction of $340 per patient per month) with a CI on the original scale.
• Robust to moderate non-normality of differences at n >= 30 (CLT).
• Straightforward CI formula; extends naturally to regression with a fixed subject effect when covariates need adjustment.
• The simplest correct choice for paired continuous data; avoids the complexity of mixed models when only two time points are present.

Cons:

• Requires complete pairs: a patient missing either the pre or post measurement must be excluded or imputed. This can introduce selection bias if missingness is not random (patients lost to follow-up often differ systematically from completers).
• Cannot handle more than two time points; for three or more repeated measurements, linear mixed models (LMM) or MMRM are the appropriate generalization.
• Blind to secular trends and regression to the mean — it faithfully estimates the mean within-person difference, but that difference may not be the causal effect of interest in an observational pre-post design.
• At small n (< 15) with non-normal differences, the Wilcoxon signed-rank test is more reliable.
• The mean difference is the target quantity; if the distribution of differences is extremely right-skewed (e.g., cost differences in patients with rare high-cost events), the mean is an unstable estimator and a gamma GLM with a subject random effect or a bootstrap CI may be preferable.

When NOT to use

• Independent groups: do not apply the paired t-test to unpaired data. Using a paired test when there is no within-subject or within-pair link inflates type-I error; use Welch's two-sample t-test instead.
• More than two time points: three or more repeated measurements require linear mixed models, MMRM, or a repeated-measures ANOVA. Reducing three time points to a single pre-post difference discards information and introduces arbitrary choices about which time point is "post."
• Skewed differences at small n: if n < 15 and the histogram of differences is clearly right-skewed or has outliers, the Wilcoxon signed-rank test is the rank-based nonparametric alternative that is valid without the normality assumption on the differences.
• Pre-post design without a concurrent control when secular trend or regression to the mean is plausible: the paired t-test will give a valid p-value for the hypothesis "the mean difference is zero" — but the observed difference may reflect time trends, seasonal effects, or regression to the mean rather than a causal effect of any intervention. Route to a difference-in-differences design or an interrupted time series if causal inference is the goal. Report the pre-post estimate as descriptive only.
• Binary or ordinal outcomes: for paired binary outcomes (e.g., readmission yes/no before and after a policy), use McNemar's test. For paired ordinal outcomes, use the Wilcoxon signed-rank test.

Interpreting the output

In the worked example, six patients enrolled in an ED diversion care management program averaged 2.0 fewer emergency department visits in the 12 months after enrollment compared with the 12 months before (mean difference d̄ = 2.0). The standard deviation of within- patient differences is 0.894, the standard error is 0.365, the t statistic is 5.48 on df = 5, the two-sided p-value is approximately 0.003, and the 95% CI on the mean paired difference is [1.06, 2.94] visits.

(1) Formal interpretation. The observed mean within-patient reduction of 2.0 visits is 5.48 standard errors above zero. Under the null hypothesis that the population mean of within-patient differences is zero, a t statistic this large on df = 5 arises by chance in approximately 0.3% of samples. The 95% CI [1.06, 2.94] is constructed so that if the study were repeated many times, approximately 95% of such intervals would contain the true mean paired difference — it is consistent with a true reduction of between about 1 and 3 visits per patient.

(2) Practical interpretation. All six patients reduced their visits after enrollment and the average reduction of 2.0 visits is statistically distinguishable from zero. However, this is a pre-post design without a concurrent control group, and these patients were enrolled specifically because of high baseline utilization. Some or all of the reduction may reflect regression to the mean rather than a program effect. A difference-in-differences design comparing this cohort to a matched unexposed group over the same period would be needed to attribute the reduction causally to the care management program.

Decision diagram

flowchart TD
  Q[Continuous outcome measured<br/>on the same subjects twice?] --> Yes["Yes — use paired design"]
  Q --> No["No — use two-sample Welch t-test<br/>or Mann-Whitney U"]
  Yes --> Diffs["Compute within-patient differences<br/>d_i = post_i - pre_i"]
  Diffs --> Check["Are differences approx normal?<br/>(histogram, Q-Q plot; n>=15)"]
  Check --> Normal["Yes: Paired t-test<br/>t = d̄ / (s_d/sqrt(n))"]
  Check --> Skewed["No (small n, skewed diffs):<br/>Wilcoxon signed-rank test"]
  Normal --> Interp["Report: mean diff + 95% CI<br/>in original units"]
  Interp --> Caution["⚠ Pre-post only design?<br/>Rule out secular trend +<br/>regression to the mean"]
  Caution --> DID["If causal inference needed:<br/>difference-in-differences<br/>or controlled pre-post"]

Worked example

Scenario

A health plan enrolled six high-utilizer patients in an emergency-department (ED) diversion care management program. An analyst wants to know whether the mean number of ED visits per patient changed after enrollment, using a paired t-test on the pre-program (12 months before) and post-program (12 months after) visit counts. The six patients had ED visit counts of 5, 3, 6, 4, 7, and 3 in the pre-period, and 3, 2, 3, 2, 4, and 2 in the post-period. The analyst computes within-patient differences, then manually walks through the paired t-test arithmetic.

Dataset

ED visits in the 12 months before and 12 months after enrollment in a care management program. The "difference" column is pre minus post for each patient.

Steps

Result

Mean difference (pre minus post) = 12/6 = 2.0 ED visits. Sum of squared deviations = 4.0; s_d^2 = 4/5 = 0.8; s_d = 0.894; SE = 0.894/2.449 = 0.365. t = 2.0/0.365 = 5.48, df = 5, p approximately 0.003 (two-sided). 95% CI: 2.0 +/- 2.571*0.365 = [1.06, 2.94] visits. Conclusion: statistically significant average reduction of 2 ED visits per patient, but a concurrent control group is needed to rule out regression to the mean before attributing the reduction to the program.

Trade-offs

Runnable example

import math
from scipy import stats

# ── Dataset: ED visits pre and post care management enrollment ──
# Six patients; pre-period and post-period 12-month visit counts.
pre  = [5, 3, 6, 4, 7, 3]
post = [3, 2, 3, 2, 4, 2]

# ── Step 1: compute within-patient differences (pre minus post) ──
diffs = [p - q for p, q in zip(pre, post)]   # [2, 1, 3, 2, 3, 1]
n = len(diffs)

# ── Step 2: mean difference ──
d_bar = sum(diffs) / n                        # 12/6 = 2.0
print(f"Differences:    {diffs}")
print(f"Sum of diffs:   {sum(diffs)}  (n={n})")
print(f"Mean diff (d̄): {d_bar:.4f}")

# ── Step 3: variance and SD of differences ──
ss = sum((d - d_bar)**2 for d in diffs)       # sum of squared deviations = 4.0
s2 = ss / (n - 1)                             # sample variance = 4/5 = 0.8
s  = math.sqrt(s2)                            # SD of diffs = sqrt(0.8) = 0.8944
se = s / math.sqrt(n)                         # SE = 0.8944/sqrt(6) = 0.3651
print(f"\nSum sq deviations: {ss:.4f}")
print(f"Variance (s²):     {s2:.4f}")
print(f"Std dev (s):       {s:.4f}")
print(f"Std error (SE):    {se:.4f}")

# ── Step 4: t-statistic and two-sided p-value (manual) ──
t_manual = d_bar / se
df = n - 1
p_manual = stats.t.sf(abs(t_manual), df=df) * 2
print(f"\nt-statistic (manual): {t_manual:.4f}")
print(f"Degrees of freedom:   {df}")
print(f"p-value (two-sided):  {p_manual:.4f}")

# ── Step 5: 95% confidence interval ──
t_crit = stats.t.ppf(0.975, df=df)           # 2.5706 for df=5
ci_lo = d_bar - t_crit * se
ci_hi = d_bar + t_crit * se
print(f"t-critical (0.975):   {t_crit:.4f}")
print(f"95% CI:               [{ci_lo:.2f}, {ci_hi:.2f}] visits")

# ── Step 6: confirm using scipy.stats.ttest_rel ──
# ttest_rel computes the paired t-test directly on the original arrays.
t_scipy, p_scipy = stats.ttest_rel(pre, post)
print(f"\nscipy.stats.ttest_rel:  t={t_scipy:.4f}, p={p_scipy:.4f}")
print("(Identical to manual calculation — confirms the implementation.)")

# ── Interpretation note ──
print(f"\nMean reduction: {d_bar:.1f} ED visits per patient (95% CI: {ci_lo:.1f}, {ci_hi:.1f})")
print("WARNING: No concurrent control — regression to the mean cannot be ruled out.")
print("         A difference-in-differences design would be needed for causal inference.")

# ── Dataset: ED visits pre and post care management enrollment ──
pre  <- c(5, 3, 6, 4, 7, 3)
post <- c(3, 2, 3, 2, 4, 2)
n    <- length(pre)

# ── Step 1: within-patient differences ──
diffs <- pre - post
cat("Differences:", diffs, "\n")
cat("Sum:", sum(diffs), "  Mean:", mean(diffs), "\n")

# ── Step 2: variance of differences (manual) ──
ss <- sum((diffs - mean(diffs))^2)
s2 <- ss / (n - 1)
s  <- sqrt(s2)
se <- s / sqrt(n)
cat(sprintf("Variance: %.4f  SD: %.4f  SE: %.4f\n", s2, s, se))
cat(sprintf("t = %.4f  df = %d\n", mean(diffs)/se, n - 1))

# ── Step 3: paired t-test via t.test ──
# paired = TRUE: t.test internally computes pre - post and runs a one-sample test.
res_paired <- t.test(pre, post, paired = TRUE, conf.level = 0.95)
cat("\n--- t.test(paired = TRUE) ---\n")
print(res_paired)

# ── Step 4: equivalence — one-sample t-test on differences ──
res_one <- t.test(diffs, mu = 0, conf.level = 0.95)
cat("\n--- t.test(diffs, mu=0) — identical result ---\n")
print(res_one)

# ── Step 5: effect size (Cohen's d_z) ──
# For paired t-test, Cohen's d_z = mean(diffs) / sd(diffs)
dz <- mean(diffs) / sd(diffs)
cat(sprintf("\nCohen's d_z (effect size for paired design): %.4f\n", dz))
cat("Interpretation: d_z >= 0.8 is a large effect by convention.\n")

# ── Wilcoxon signed-rank as a sensitivity check ──
wsr <- wilcox.test(pre, post, paired = TRUE, exact = FALSE)
cat(sprintf("\nWilcoxon signed-rank (sensitivity): W=%.1f, p=%.4f\n",
            wsr$statistic, wsr$p.value))
cat("If p-values from paired t-test and Wilcoxon are consistent, the normality\n")
cat("assumption is not load-bearing for this dataset.\n")

/* ── Create dataset: ED visits pre and post enrollment ── */
data work.ed_visits;
  input patient_id $ pre post;
  diff = pre - post;     /* within-patient difference: pre minus post */
  datalines;
P1 5 3
P2 3 2
P3 6 3
P4 4 2
P5 7 4
P6 3 2
;
run;

/* ── Verify differences manually ── */
proc means data=work.ed_visits n mean std stderr;
  var diff;
  title 'Descriptives for within-patient differences (pre - post)';
run;

/* ── Paired t-test: Method 1 — PROC TTEST with PAIRED statement ──
   SAS reads the two variables directly and computes within-pair differences.
   The "pre - post" notation specifies the direction of subtraction.          */
proc ttest data=work.ed_visits alpha=0.05;
  paired pre*post;          /* tests whether mean(pre - post) = 0             */
  title 'Paired t-test: pre vs post ED visits (PROC TTEST PAIRED)';
run;
/* Key output rows:
   - "Difference" row: mean difference, 95% CI, t-statistic, DF, p-value
   - Satterthwaite and Pooled rows are for the equivalent two-sample comparison;
     use the PAIRED output row for the actual paired test result.             */

/* ── Paired t-test: Method 2 — one-sample PROC TTEST on the diff variable ──
   Equivalent to Method 1; confirms the paired t-test is a one-sample test
   on the within-patient differences with H0: mu_diff = 0.                   */
proc ttest data=work.ed_visits h0=0 sides=2;
  var diff;
  title 'One-sample t-test on differences (equivalent to paired t-test)';
run;

/* ── Wilcoxon signed-rank test as nonparametric sensitivity check ──
   PROC UNIVARIATE prints the Wilcoxon signed-rank statistic and p-value.    */
proc univariate data=work.ed_visits;
  var diff;
  title 'Wilcoxon signed-rank sensitivity check on within-patient differences';
run;
/* Report both the paired t-test p-value and the Wilcoxon signed-rank p-value.
   If they agree, the normality assumption is not load-bearing for inference. */

Citations

1. [1]Zimmerman DW. A note on interpretation of the paired-samples t test. Journal of Educational and Behavioral Statistics. 1997;22(3):349-360.
2. [2]Bland JM, Altman DG. Statistics notes: Matching. BMJ. 1994;309(6962):1128.

1. [3]Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology. 2013;4:863.

1. [4]Zimmerman DW. A note on interpretation of the paired-samples t test. Journal of Educational and Behavioral Statistics. 1997;22(3):349-360.