McNemar's Test for Paired Proportions

In plain language

McNemar's test answers one question: did the fraction of patients with a "yes" outcome change between two conditions measured on the same people? You pair each patient's before-and-after result (or case-and-control result) in a 2x2 table, then the test looks only at the pairs where the two conditions disagreed — because pairs where both conditions gave the same answer carry no information about whether anything changed. It produces a p-value for whether the two proportions differ, but it cannot tell you why they differ or whether the change was caused by the intervention.

The core insight: only discordant pairs carry information

When you measure the same binary outcome on the same subjects under two conditions — before and after a program, case versus matched control, two diagnostic tests on the same patient — you construct a paired 2x2 table with four cells:

• a: both conditions positive (concordant)
• b: condition 1 positive, condition 2 negative (discordant)
• c: condition 1 negative, condition 2 positive (discordant)
• d: both conditions negative (concordant)

The concordant cells (a and d) are completely uninformative about whether the two marginal proportions differ. If a patient was hospitalized both before and after the program, that tells you nothing about the program's effect on hospitalization rates. Only the discordant cells b and c carry signal: b counts patients whose status flipped from positive to negative, and c counts patients whose status flipped from negative to positive. Under the null hypothesis that the two marginal proportions are equal (that is, that the probability of "yes" is the same under both conditions), the discordant count b should equal c in expectation — formally, b/(b+c) should follow a Binomial(b+c, 0.5) distribution. McNemar's test formalizes this.

The test statistic and exact vs asymptotic versions

The asymptotic McNemar statistic is:

chi-square = (b - c)^2 / (b + c)

which follows a chi-squared distribution with 1 degree of freedom under the null when b + c is reasonably large (typically b + c >= 25 is cited as a rule of thumb). For small discordant totals, the exact version based directly on the Binomial(b+c, 0.5) distribution is preferred — this gives the exact probability of observing a split as extreme or more extreme than b vs c if the true probability were 0.5. The exact test is conservative (it uses the discrete probability mass function, yielding slightly elevated type-I error protection at the cost of power), and Fagerland, Lydersen, and Laake (2013) demonstrated that the mid-p correction — which adds half the probability of the observed table to each tail — achieves better balance between type-I and type-II error rates than either the exact or asymptotic test in most practical situations. The mid-p approach is now the recommended default for small b + c.

Effect estimation alongside the p-value

A p-value alone is insufficient for decision-making. The natural effect estimate paired with McNemar's test is the marginal proportion difference: the proportion who are positive under condition 1 minus the proportion positive under condition 2, which equals (b - c) / n, where n is the total number of pairs. An equivalent framing is the ratio of discordant proportions (b/c or its inverse), sometimes called the McNemar odds ratio. Confidence intervals for the marginal proportion difference can be obtained by the exact binomial approach or by asymptotic normal approximation; the exact2x2 package in R is a convenient implementation.

RWE applications: before-after binary indicators within patients

In real-world evidence, McNemar's test arises naturally in three settings:

1. Pre-post comparison within the same patients: Measure a binary outcome (any hospitalization yes/no, medication possession ratio >= 80% yes/no, ACE inhibitor use yes/no) in a fixed window before and after an index event (diagnosis, treatment initiation, policy change). Each patient contributes one pair of observations. The test asks whether the proportion positive shifted between the two windows — for example, whether the fraction of patients with a hospitalization was higher in the year before versus the year after starting a disease management program.

2. Matched case-control pairs: In a 1:1 matched case-control study with a binary exposure, the exposure status of the case and the matched control form a pair. The McNemar test is equivalent to a conditional logistic regression with one predictor in this setting, and conditional logistic regression is the appropriate multivariable extension when you need to adjust for additional covariates after matching. Understanding McNemar as the special case of conditional logistic regression without covariates clarifies why ordinary logistic regression is wrong for paired data — it ignores the matching structure.

3. Agreement versus marginal change (do not confuse): McNemar's test measures marginal homogeneity — whether the row margin (proportion positive under condition 1) equals the column margin (proportion positive under condition 2). It does NOT measure agreement. Cohen's kappa measures the degree to which two raters or two tests agree beyond chance, which is a different question entirely. Two raters could systematically disagree (one always says "yes," one always says "no") yielding a large McNemar statistic and kappa = -1. They could also mostly agree while disagreeing asymmetrically enough to produce a significant McNemar p-value. Always clarify which question — marginal change or agreement — your analysis addresses.

The self-controlled flavor and its limitations

The pre-post McNemar design is a form of self-controlled analysis: each patient serves as their own control, which eliminates confounding from all time-fixed patient characteristics (demographics, baseline comorbidities, health-seeking behavior that is stable over time). This is a powerful design feature — no propensity score or external comparator group is needed to eliminate fixed confounders. However, the self-controlled structure does NOT eliminate time-varying confounding. If the outcome probability would have changed over time in the absence of the intervention due to disease progression, seasonal effects, secular trends, or regression to the mean, the pre-post comparison is biased even for the same patients. Similarly, the McNemar design does not adjust for the duration of follow-up if patients have different amounts of time in each window. These limitations connect directly to the broader catalog of self-controlled designs (self-controlled case series, SCCS) where conditioning on cases only and modeling the timing of exposure within patients addresses some of these concerns more rigorously than a simple pre-post binary test.

Pros, cons, and trade-offs

Pros: - Conceptually elegant: focuses inference precisely on the discordant pairs, which are the only pairs carrying information about marginal change. - Efficient: uses within-patient pairing to eliminate fixed confounding without an external comparator, reducing the variance of the estimator compared to an independent-sample chi-square on the same data. - Three well-validated versions (exact, asymptotic, mid-p) with clear guidance on which to use by discordant count. - Direct extension to conditional logistic regression for multivariable adjustment after 1:1 matching. - Computationally trivial — available in every major statistical software package.

Cons: - Discards the concordant pairs entirely — a large dataset with mostly concordant pairs yields very low power, because effective n is only b + c, not the total number of pairs. - For small b + c the power is genuinely low regardless of total sample size; no amount of enrolling more concordant pairs compensates. - Tests only marginal homogeneity; cannot directly address whether the change is caused by the intervention. - Cannot adjust for covariates directly — must graduate to conditional logistic regression if covariate adjustment is needed. - Assumes binary outcome; the multi-category generalization (Stuart-Maxwell test) is less widely known and implemented. - Pre-post McNemar does not adjust for time-varying confounders or secular trends.

When to use

Use McNemar's test when:

• The outcome is binary and measurements come in natural pairs: the same patient
• The scientific question is whether the proportion positive changed between
• You want a formal inference about the marginal shift alongside the visual or
• The discordant count b + c is at least 5 (exact or mid-p test) or at least 25
• As a quick diagnostic in matched case-control studies before graduating to
• Sensitivity analysis: after 1:1 propensity-score matching, McNemar's test on a

When NOT to use — and when these tests are actively misleading

• Independent groups: if the two groups of patients are different people (treated
• More than two categories: McNemar's test is for 2x2 paired tables only. For an
• Recurrent events or count outcomes: McNemar's test requires a binary yes/no per
• When time-varying confounding dominates the pre-post contrast: if there is
• Confusing McNemar with Cohen's kappa: McNemar tests marginal homogeneity;
• When the total discordant count b + c is very small (< 5): power is negligible.

Implementation note for all three languages

The canonical demonstration below tests whether the proportion of patients with any hospitalization differed in the year before versus the year after a chronic disease diagnosis — a pre-post RWE use case using the same paired 2x2 table from the beginner layer. All three implementations show both the asymptotic chi-square version and the exact/mid-p version, with the discordant-pair framing explicit in comments. The SAS implementation uses PROC FREQ with the AGREE option and the TEST MCNEM statement.

Interpreting the output

In the worked example, 40 patients each have a binary hospitalization indicator for the year before and after a chronic disease diagnosis. Of the 20 discordant pairs, 15 flipped from hospitalized to not hospitalized (b = 15) and 5 flipped the other direction (c = 5). The McNemar statistic is (b − c)² / (b + c) = 100 / 20 = 5 on df = 1, with p ≈ 0.025. The pre-period hospitalization proportion was 25/40 = 0.625 and the post-period proportion was 15/40 = 0.375, a marginal difference of 0.25.

(1) Formal interpretation. The McNemar statistic of 5 on df = 1 has a p-value of approximately 0.025 under the null hypothesis that discordant pairs are equally likely to flip in either direction — that is, that b/(b + c) = 0.5. This falls below the conventional alpha = 0.05 threshold. The entire computation rests on the 20 discordant pairs; the 20 concordant pairs (patients hospitalized in both windows, or neither window) carry no information about marginal change and are excluded. The marginal proportion difference of 0.25 (25 fewer hospitalizations per 100 patients) is the natural accompanying effect estimate.

(2) Practical interpretation. The proportion of patients with any hospitalization dropped by about 25 percentage points between the two measurement windows, and this shift is statistically distinguishable from chance at conventional levels. Because each patient serves as their own control, time-fixed confounders such as baseline comorbidities are eliminated — but time-varying factors including disease progression, secular trends, and regression to the mean could still explain some or all of the reduction. The 3:1 imbalance among discordant pairs (15 improved, 5 worsened) drives the result; concordant pairs contribute nothing to the test statistic regardless of how numerous they are.

Worked example

Scenario

A health outcomes analyst is studying whether any-hospitalization rates changed in the year before versus the year after a chronic disease diagnosis for 40 patients. Each patient has exactly one pre-period outcome (hospitalized yes/no in the year before diagnosis) and one post-period outcome (hospitalized yes/no in the year after diagnosis). The analyst builds the paired 2x2 table and runs McNemar's test to decide whether the proportion hospitalized shifted significantly.

Dataset

Paired 2x2 table: rows = pre-diagnosis hospitalization status, columns = post-diagnosis hospitalization status. Cell counts are the number of patient pairs in each category. Total pairs n = 40.

Steps

• Identify the four cells. Concordant pairs where both periods agree: cell a = 10 (hospitalized both years) and cell d = 10 (not hospitalized either year). Discordant pairs where the periods disagree: cell b = 15 (hospitalized pre, not post) and cell c = 5 (not hospitalized pre, hospitalized post).

• Compute marginal proportions. Pre-period proportion hospitalized = row total for 'Yes pre' / n = 25/40 = 0.625. Post-period proportion hospitalized = column total for 'Yes post' / n = 15/40 = 0.375. The apparent shift is 0.625 - 0.375 = 0.25.

• The concordant pairs (a = 10 and d = 10) carry NO information about whether the proportions changed. Discard them. The test is built entirely on the discordant total: b + c = 15 + 5 = 20.

• Compute the asymptotic McNemar statistic: chi-square equals the squared difference of the discordant counts divided by their sum. The numerator is (b - c) squared = (15 - 5) squared = 10 squared = 100. The denominator is b + c = 15 + 5 = 20. So chi-square = 100 / 20 = 5. Compare to a chi-squared distribution with 1 degree of freedom.

• A chi-squared value of 5 with 1 df gives a p-value of approximately 0.025 (two-sided). This is below the conventional alpha = 0.05 threshold, so we reject the null hypothesis that the two marginal proportions are equal.

• Interpret in context. The proportion hospitalized was 25/40 = 0.625 before diagnosis and 15/40 = 0.375 after diagnosis. The marginal proportion difference is 0.625 - 0.375 = 0.25, meaning about 25 percentage points fewer patients had any hospitalization in the year after diagnosis than in the year before. Whether this represents a causal effect of post-diagnosis management or reflects natural disease course and regression to the mean requires a design with a comparator group.

Result

Discordant pairs b = 15, c = 5; discordant total b + c = 15 + 5 = 20. Numerator = (b - c) squared = 10 squared = 100. McNemar statistic = 100 / 20 = 5; p approximately 0.025 (two-sided, 1 df). Pre-period proportion hospitalized = 25 / 40 = 0.625; post-period proportion = 15 / 40 = 0.375; marginal proportion difference = 0.625 - 0.375 = 0.25. The test is significant at alpha = 0.05. The concordant pairs (a = 10, d = 10) were discarded because they carry zero information about the marginal shift.

Runnable example

import numpy as np
from statsmodels.stats.contingency_tables import mcnemar

# ── Paired 2x2 table from the worked example ──
# Rows: pre-period (Yes=1, No=0); Columns: post-period (Yes=1, No=0)
# Layout: [[a, b], [c, d]]  where b and c are the discordant cells
# a=10: both hospitalized; b=15: pre-yes, post-no; c=5: pre-no, post-yes; d=10: both not
table = np.array([[10, 15],
                  [ 5, 10]])

a, b, c, d = table[0, 0], table[0, 1], table[1, 0], table[1, 1]
n = a + b + c + d  # total pairs

# ── Marginal proportions ──
prop_pre  = (a + b) / n   # proportion positive pre-period
prop_post = (a + c) / n   # proportion positive post-period
marg_diff = prop_pre - prop_post

print(f"Total pairs n = {n}")
print(f"Concordant pairs: a={a} (both yes), d={d} (both no)")
print(f"Discordant pairs: b={b} (pre-yes, post-no), c={c} (pre-no, post-yes)")
print(f"Discordant total b+c = {b+c}")
print(f"Pre-period proportion: {a+b}/{n} = {prop_pre:.4f}")
print(f"Post-period proportion: {a+c}/{n} = {prop_post:.4f}")
print(f"Marginal proportion difference (pre - post): {marg_diff:.4f}")

# ── Asymptotic McNemar: (b-c)^2 / (b+c) ~ chi-squared(1) ──
# Recommended when b+c >= 25; here b+c = 20 so asymptotic is borderline.
result_asymptotic = mcnemar(table, exact=False, correction=False)
print(f"\nAsymptotic McNemar: statistic={result_asymptotic.statistic:.4f}, "
      f"p={result_asymptotic.pvalue:.4f}")
print(f"  Manual check: (b-c)^2/(b+c) = ({b}-{c})^2/({b}+{c}) "
      f"= {(b-c)**2}/{b+c} = {(b-c)**2/(b+c):.4f}")

# ── Exact McNemar: binomial(b+c, 0.5) exact test ──
# Preferred when b+c < 25; conservative (slight upward bias in p-value).
result_exact = mcnemar(table, exact=True)
print(f"\nExact McNemar:      statistic={result_exact.statistic:.4f}, "
      f"p={result_exact.pvalue:.4f}")

# ── Mid-p McNemar: Fagerland et al. (2013) recommendation for small b+c ──
# statsmodels does not implement mid-p directly; approximation via scipy binom
from scipy.stats import binom
# Two-sided exact p: P(X <= min(b,c)) + P(X >= max(b,c)) where X ~ Binom(b+c, 0.5)
m = b + c
x_obs = min(b, c)
exact_p = 2 * binom.cdf(x_obs, m, 0.5)
midp_p = exact_p - binom.pmf(x_obs, m, 0.5)  # subtract half the probability of observed
print(f"\nMid-p McNemar:      p={midp_p:.4f}")
print("  (Fagerland et al. 2013 recommend mid-p for b+c < 25)")

# ── Confidence interval for marginal proportion difference (asymptotic) ──
import math
se_diff = math.sqrt((b + c - (b - c)**2 / n) / n**2)
ci_lo = marg_diff - 1.96 * se_diff
ci_hi = marg_diff + 1.96 * se_diff
print(f"\n95% CI for marginal proportion difference: "
      f"({ci_lo:.4f}, {ci_hi:.4f})")

# ── Key interpretation reminder ──
print("\nNote: concordant pairs (a, d) are discarded by the test.")
print("McNemar tests marginal homogeneity, NOT agreement (use Cohen's kappa for agreement).")
print("For multivariable adjustment after 1:1 matching, use conditional logistic regression.")

# ── Paired 2x2 table: rows = pre-period, columns = post-period ──
# [[a, b], [c, d]] — discordant cells are b (row 1, col 2) and c (row 2, col 1)
table_2x2 <- matrix(c(10, 15, 5, 10), nrow = 2, byrow = TRUE,
                    dimnames = list(Pre  = c("Yes", "No"),
                                    Post = c("Yes", "No")))
print("Paired 2x2 table:")
print(table_2x2)

a <- table_2x2[1, 1]; b <- table_2x2[1, 2]
c <- table_2x2[2, 1]; d <- table_2x2[2, 2]
n <- a + b + c + d

cat(sprintf("\nTotal pairs n = %d\n", n))
cat(sprintf("Discordant pairs: b=%d (pre-yes, post-no), c=%d (pre-no, post-yes)\n", b, c))
cat(sprintf("Discordant total b+c = %d\n", b + c))

# ── Marginal proportions ──
prop_pre  <- (a + b) / n
prop_post <- (a + c) / n
cat(sprintf("\nPre-period proportion:  (%d+%d)/%d = %.4f\n", a, b, n, prop_pre))
cat(sprintf("Post-period proportion: (%d+%d)/%d = %.4f\n", a, c, n, prop_post))
cat(sprintf("Marginal proportion difference (pre - post): %.4f\n", prop_pre - prop_post))

# ── Asymptotic McNemar: (b-c)^2/(b+c) ~ chisq(1) ──
# correct=FALSE: no continuity correction (standard for asymptotic McNemar)
res_asymptotic <- mcnemar.test(table_2x2, correct = FALSE)
cat("\nAsymptotic McNemar (correct=FALSE):\n")
print(res_asymptotic)
cat(sprintf("  Manual check: (b-c)^2/(b+c) = (%d-%d)^2/(%d+%d) = %d/%d = %.4f\n",
            b, c, b, c, (b - c)^2, b + c, (b - c)^2 / (b + c)))

# ── Exact McNemar: base R uses exact binomial when correct=TRUE and ties present ──
# For clarity, compute directly with binom.test
res_exact_binom <- binom.test(min(b, c), b + c, p = 0.5)
cat(sprintf("\nExact McNemar (via binom.test): p = %.4f\n", res_exact_binom$p.value))

# ── Mid-p McNemar: Fagerland et al. (2013) — requires exact2x2 package ──
# Install once: install.packages("exact2x2")
# library(exact2x2)
# res_midp <- mcnemar.exact(table_2x2, midp = TRUE)
# cat(sprintf("Mid-p McNemar: p = %.4f\n", res_midp$p.value))
cat("\nFor mid-p correction (recommended when b+c < 25), use:\n")
cat("  library(exact2x2); mcnemar.exact(table_2x2, midp = TRUE)\n")
cat("  [exact2x2 package — Fagerland et al. 2013]\n")

# ── Interpretation notes ──
cat("\nKey reminders:\n")
cat("  - Concordant pairs (a, d) are IGNORED by the test.\n")
cat("  - McNemar tests marginal homogeneity; kappa tests agreement.\n")
cat("  - For multivariable adjustment after 1:1 matching:\n")
cat("    library(survival); clogit(outcome ~ exposure + strata(pair_id))\n")