McNemar's Test for Paired Proportions

The core insight: only discordant pairs carry information

When you measure the same binary outcome on the same subjects under two conditions — before and after a program, case versus matched control, two diagnostic tests on the same patient — you construct a paired 2x2 table with four cells:

• a: both conditions positive (concordant)
• b: condition 1 positive, condition 2 negative (discordant)
• c: condition 1 negative, condition 2 positive (discordant)
• d: both conditions negative (concordant)

The concordant cells (a and d) are completely uninformative about whether the two marginal proportions differ. If a patient was hospitalized both before and after the program, that tells you nothing about the program's effect on hospitalization rates. Only the discordant cells b and c carry signal: b counts patients whose status flipped from positive to negative, and c counts patients whose status flipped from negative to positive.

Under the null hypothesis that the two marginal proportions are equal (that is, that the probability of "yes" is the same under both conditions), the discordant count b should equal c in expectation — formally, b/(b+c) should follow a Binomial(b+c, 0.5) distribution.

McNemar's test formalizes this.

The test statistic and exact vs asymptotic versions

The asymptotic McNemar statistic is:

chi-square = (b - c)^2 / (b + c)

which follows a chi-squared distribution with 1 degree of freedom under the null when b + c is reasonably large (typically b + c >= 25 is cited as a rule of thumb). For small discordant totals, the exact version based directly on the Binomial(b+c, 0.5) distribution is preferred — this gives the exact probability of observing a split as extreme or more extreme than b vs c if the true probability were 0.5.

The exact test is conservative (it uses the discrete probability mass function, yielding slightly elevated type-I error protection at the cost of power), and Fagerland, Lydersen, and Laake (2013) demonstrated that the mid-p correction — which adds half the probability of the observed table to each tail — achieves better balance between type-I and type-II error rates than either the exact or asymptotic test in most practical situations.

The mid-p approach is now the recommended default for small b + c.

Effect estimation alongside the p-value

A p-value alone is insufficient for decision-making. The natural effect estimate paired with McNemar's test is the marginal proportion difference: the proportion who are positive under condition 1 minus the proportion positive under condition 2, which equals (b - c) / n, where n is the total number of pairs.

An equivalent framing is the ratio of discordant proportions (b/c or its inverse), sometimes called the McNemar odds ratio. Confidence intervals for the marginal proportion difference can be obtained by the exact binomial approach or by asymptotic normal approximation; the exact2x2 package in R is a convenient implementation.

RWE applications: before-after binary indicators within patients

In real-world evidence, McNemar's test arises naturally in three settings:

1. Pre-post comparison within the same patients: Measure a binary outcome (any hospitalization yes/no, medication possession ratio >= 80% yes/no, ACE inhibitor use yes/no) in a fixed window before and after an index event (diagnosis, treatment initiation, policy change). Each patient contributes one pair of observations. The test asks whether the proportion positive shifted between the two windows — for example, whether the fraction of patients with a hospitalization was higher in the year before versus the year after starting a disease management program.
2. Matched case-control pairs: In a 1:1 matched case-control study with a binary exposure, the exposure status of the case and the matched control form a pair. The McNemar test is equivalent to a conditional logistic regression with one predictor in this setting, and conditional logistic regression is the appropriate multivariable extension when you need to adjust for additional covariates after matching. Understanding McNemar as the special case of conditional logistic regression without covariates clarifies why ordinary logistic regression is wrong for paired data — it ignores the matching structure.
3. Agreement versus marginal change (do not confuse): McNemar's test measures marginal homogeneity — whether the row margin (proportion positive under condition 1) equals the column margin (proportion positive under condition 2). It does NOT measure agreement. Cohen's kappa measures the degree to which two raters or two tests agree beyond chance, which is a different question entirely. Two raters could systematically disagree (one always says "yes," one always says "no") yielding a large McNemar statistic and kappa = -1. They could also mostly agree while disagreeing asymmetrically enough to produce a significant McNemar p-value. Always clarify which question — marginal change or agreement — your analysis addresses.

The self-controlled flavor and its limitations

The pre-post McNemar design is a form of self-controlled analysis: each patient serves as their own control, which eliminates confounding from all time-fixed patient characteristics (demographics, baseline comorbidities, health-seeking behavior that is stable over time). This is a powerful design feature — no propensity score or external comparator group is needed to eliminate fixed confounders.

However, the self-controlled structure does NOT eliminate time-varying confounding. If the outcome probability would have changed over time in the absence of the intervention due to disease progression, seasonal effects, secular trends, or regression to the mean, the pre-post comparison is biased even for the same patients.

Similarly, the McNemar design does not adjust for the duration of follow-up if patients have different amounts of time in each window. These limitations connect directly to the broader catalog of self-controlled designs (self-controlled case series, SCCS) where conditioning on cases only and modeling the timing of exposure within patients addresses some of these concerns more rigorously than a simple pre-post binary test.

Pros, cons, and trade-offs

Pros:

• Conceptually elegant: focuses inference precisely on the discordant pairs, which are the only pairs carrying information about marginal change.
• Efficient: uses within-patient pairing to eliminate fixed confounding without an external comparator, reducing the variance of the estimator compared to an independent-sample chi-square on the same data.
• Three well-validated versions (exact, asymptotic, mid-p) with clear guidance on which to use by discordant count.
• Direct extension to conditional logistic regression for multivariable adjustment after 1:1 matching.
• Computationally trivial — available in every major statistical software package.

Cons:

• Discards the concordant pairs entirely — a large dataset with mostly concordant pairs yields very low power, because effective n is only b + c, not the total number of pairs.
• For small b + c the power is genuinely low regardless of total sample size; no amount of enrolling more concordant pairs compensates.
• Tests only marginal homogeneity; cannot directly address whether the change is caused by the intervention.
• Cannot adjust for covariates directly — must graduate to conditional logistic regression if covariate adjustment is needed.
• Assumes binary outcome; the multi-category generalization (Stuart-Maxwell test) is less widely known and implemented.
• Pre-post McNemar does not adjust for time-varying confounders or secular trends.

When NOT to use — and when these tests are actively misleading

• Independent groups: if the two groups of patients are different people (treated vs untreated cohorts without 1:1 matching), McNemar's test is wrong — the pairing structure it assumes does not exist. Use the chi-square test or Fisher's exact test instead. Applying McNemar to independent samples produces incorrect p-values.
• More than two categories: McNemar's test is for 2x2 paired tables only. For an ordinal or multinomial paired outcome (severity grade: none/mild/moderate/severe before and after), use the Stuart-Maxwell test or Bowker's test of symmetry.
• Recurrent events or count outcomes: McNemar's test requires a binary yes/no per person per period. If the outcome is the number of hospitalizations (which can be 0, 1, 2, ...) you need a paired count model — a negative binomial mixed model or a paired Poisson approach — not a test for binary proportions.
• When time-varying confounding dominates the pre-post contrast: if there is strong evidence that disease progression, a co-occurring policy change, or secular trends would change the outcome probability over the same window regardless of the intervention, the pre-post McNemar comparison is biased. In these settings, a difference-in-differences design with an external comparator group, or an SCCS analysis, is more appropriate.
• Confusing McNemar with Cohen's kappa: McNemar tests marginal homogeneity; kappa tests agreement. Using McNemar to assess whether two diagnostic tests "agree" answers the wrong question. Using kappa to test whether a pre-post proportion changed is also wrong. Be explicit about which question is the target.
• When the total discordant count b + c is very small (< 5): power is negligible. Report the discordant pair counts descriptively and acknowledge the test is underpowered rather than accepting a non-significant p-value as evidence of no change.

Implementation note for all three languages

The canonical demonstration below tests whether the proportion of patients with any hospitalization differed in the year before versus the year after a chronic disease diagnosis — a pre-post RWE use case using the same paired 2x2 table from the beginner layer. All three implementations show both the asymptotic chi-square version and the exact/mid-p version, with the discordant-pair framing explicit in comments. The SAS implementation uses PROC FREQ with the AGREE option and the TEST MCNEM statement.

Interpreting the output

In the worked example, 40 patients each have a binary hospitalization indicator for the year before and after a chronic disease diagnosis. Of the 20 discordant pairs, 15 flipped from hospitalized to not hospitalized (b = 15) and 5 flipped the other direction (c = 5).

The McNemar statistic is (b − c)² / (b + c) = 100 / 20 = 5 on df = 1, with p ≈ 0.025. The pre-period hospitalization proportion was 25/40 = 0.625 and the post-period proportion was 15/40 = 0.375, a marginal difference of 0.25.

(1) Formal interpretation. The McNemar statistic of 5 on df = 1 has a p-value of approximately 0.025 under the null hypothesis that discordant pairs are equally likely to flip in either direction — that is, that b/(b + c) = 0.5. This falls below the conventional alpha = 0.05 threshold.

The entire computation rests on the 20 discordant pairs; the 20 concordant pairs (patients hospitalized in both windows, or neither window) carry no information about marginal change and are excluded. The marginal proportion difference of 0.25 (25 fewer hospitalizations per 100 patients) is the natural accompanying effect estimate.

(2) Practical interpretation. The proportion of patients with any hospitalization dropped by about 25 percentage points between the two measurement windows, and this shift is statistically distinguishable from chance at conventional levels. Because each patient serves as their own control, time-fixed confounders such as baseline comorbidities are eliminated — but time-varying factors including disease progression, secular trends, and regression to the mean could still explain some or all of the reduction.

The 3:1 imbalance among discordant pairs (15 improved, 5 worsened) drives the result; concordant pairs contribute nothing to the test statistic regardless of how numerous they are.

Decision diagram

flowchart TD
  Start["Binary outcome on SAME patients<br/>under two conditions?"] -->|Yes| Pair["Build paired 2x2 table<br/>(rows = condition 1, cols = condition 2)"]
  Start -->|No - independent groups| ChiSq["Use chi-square test<br/>or Fisher exact test"]
  Pair --> Count["Count discordant pairs<br/>b (cond1=Yes, cond2=No)<br/>c (cond1=No, cond2=Yes)"]
  Count --> Disc["Discordant total<br/>b + c = ?"]
  Disc -->|"b + c >= 25"| Asymp["Asymptotic McNemar<br/>(b-c)^2 / (b+c) ~ χ²(1)"]
  Disc -->|"b + c < 25"| SmallN["Small discordant total"]
  SmallN --> MidP["Mid-p McNemar<br/>(Fagerland et al. 2013)<br/>Recommended default"]
  SmallN --> Exact["Exact McNemar<br/>(Binomial test)<br/>Conservative"]
  Asymp --> Report["Report:\n- McNemar statistic + p-value\n- Marginal proportion difference\n- 95% CI"]
  MidP --> Report
  Exact --> Report
  Report --> Adjust["Need covariate adjustment?"]
  Adjust -->|Yes| CLR["Use conditional logistic regression<br/>(multivariable extension of McNemar)"]
  Adjust -->|No| Done["McNemar test is sufficient"]

Worked example

Scenario

A health outcomes analyst is studying whether any-hospitalization rates changed in the year before versus the year after a chronic disease diagnosis for 40 patients. Each patient has exactly one pre-period outcome (hospitalized yes/no in the year before diagnosis) and one post-period outcome (hospitalized yes/no in the year after diagnosis). The analyst builds the paired 2x2 table and runs McNemar's test to decide whether the proportion hospitalized shifted significantly.

Dataset

Paired 2x2 table: rows = pre-diagnosis hospitalization status, columns = post-diagnosis hospitalization status. Cell counts are the number of patient pairs in each category. Total pairs n = 40.

Steps

Result

Discordant pairs b = 15, c = 5; discordant total b + c = 15 + 5 = 20. Numerator = (b - c) squared = 10 squared = 100. McNemar statistic = 100 / 20 = 5; p approximately 0.025 (two-sided, 1 df). Pre-period proportion hospitalized = 25 / 40 = 0.625; post-period proportion = 15 / 40 = 0.375; marginal proportion difference = 0.625 - 0.375 = 0.25. The test is significant at alpha = 0.05. The concordant pairs (a = 10, d = 10) were discarded because they carry zero information about the marginal shift.

Trade-offs

Runnable example

import numpy as np
from statsmodels.stats.contingency_tables import mcnemar

# ── Paired 2x2 table from the worked example ──
# Rows: pre-period (Yes=1, No=0); Columns: post-period (Yes=1, No=0)
# Layout: [[a, b], [c, d]]  where b and c are the discordant cells
# a=10: both hospitalized; b=15: pre-yes, post-no; c=5: pre-no, post-yes; d=10: both not
table = np.array([[10, 15],
                  [ 5, 10]])

a, b, c, d = table[0, 0], table[0, 1], table[1, 0], table[1, 1]
n = a + b + c + d  # total pairs

# ── Marginal proportions ──
prop_pre  = (a + b) / n   # proportion positive pre-period
prop_post = (a + c) / n   # proportion positive post-period
marg_diff = prop_pre - prop_post

print(f"Total pairs n = {n}")
print(f"Concordant pairs: a={a} (both yes), d={d} (both no)")
print(f"Discordant pairs: b={b} (pre-yes, post-no), c={c} (pre-no, post-yes)")
print(f"Discordant total b+c = {b+c}")
print(f"Pre-period proportion: {a+b}/{n} = {prop_pre:.4f}")
print(f"Post-period proportion: {a+c}/{n} = {prop_post:.4f}")
print(f"Marginal proportion difference (pre - post): {marg_diff:.4f}")

# ── Asymptotic McNemar: (b-c)^2 / (b+c) ~ chi-squared(1) ──
# Recommended when b+c >= 25; here b+c = 20 so asymptotic is borderline.
result_asymptotic = mcnemar(table, exact=False, correction=False)
print(f"\nAsymptotic McNemar: statistic={result_asymptotic.statistic:.4f}, "
      f"p={result_asymptotic.pvalue:.4f}")
print(f"  Manual check: (b-c)^2/(b+c) = ({b}-{c})^2/({b}+{c}) "
      f"= {(b-c)**2}/{b+c} = {(b-c)**2/(b+c):.4f}")

# ── Exact McNemar: binomial(b+c, 0.5) exact test ──
# Preferred when b+c < 25; conservative (slight upward bias in p-value).
result_exact = mcnemar(table, exact=True)
print(f"\nExact McNemar:      statistic={result_exact.statistic:.4f}, "
      f"p={result_exact.pvalue:.4f}")

# ── Mid-p McNemar: Fagerland et al. (2013) recommendation for small b+c ──
# statsmodels does not implement mid-p directly; approximation via scipy binom
from scipy.stats import binom
# Two-sided exact p: P(X <= min(b,c)) + P(X >= max(b,c)) where X ~ Binom(b+c, 0.5)
m = b + c
x_obs = min(b, c)
exact_p = 2 * binom.cdf(x_obs, m, 0.5)
midp_p = exact_p - binom.pmf(x_obs, m, 0.5)  # subtract half the probability of observed
print(f"\nMid-p McNemar:      p={midp_p:.4f}")
print("  (Fagerland et al. 2013 recommend mid-p for b+c < 25)")

# ── Confidence interval for marginal proportion difference (asymptotic) ──
import math
se_diff = math.sqrt((b + c - (b - c)**2 / n) / n**2)
ci_lo = marg_diff - 1.96 * se_diff
ci_hi = marg_diff + 1.96 * se_diff
print(f"\n95% CI for marginal proportion difference: "
      f"({ci_lo:.4f}, {ci_hi:.4f})")

# ── Key interpretation reminder ──
print("\nNote: concordant pairs (a, d) are discarded by the test.")
print("McNemar tests marginal homogeneity, NOT agreement (use Cohen's kappa for agreement).")
print("For multivariable adjustment after 1:1 matching, use conditional logistic regression.")

# ── Paired 2x2 table: rows = pre-period, columns = post-period ──
# [[a, b], [c, d]] — discordant cells are b (row 1, col 2) and c (row 2, col 1)
table_2x2 <- matrix(c(10, 15, 5, 10), nrow = 2, byrow = TRUE,
                    dimnames = list(Pre  = c("Yes", "No"),
                                    Post = c("Yes", "No")))
print("Paired 2x2 table:")
print(table_2x2)

a <- table_2x2[1, 1]; b <- table_2x2[1, 2]
c <- table_2x2[2, 1]; d <- table_2x2[2, 2]
n <- a + b + c + d

cat(sprintf("\nTotal pairs n = %d\n", n))
cat(sprintf("Discordant pairs: b=%d (pre-yes, post-no), c=%d (pre-no, post-yes)\n", b, c))
cat(sprintf("Discordant total b+c = %d\n", b + c))

# ── Marginal proportions ──
prop_pre  <- (a + b) / n
prop_post <- (a + c) / n
cat(sprintf("\nPre-period proportion:  (%d+%d)/%d = %.4f\n", a, b, n, prop_pre))
cat(sprintf("Post-period proportion: (%d+%d)/%d = %.4f\n", a, c, n, prop_post))
cat(sprintf("Marginal proportion difference (pre - post): %.4f\n", prop_pre - prop_post))

# ── Asymptotic McNemar: (b-c)^2/(b+c) ~ chisq(1) ──
# correct=FALSE: no continuity correction (standard for asymptotic McNemar)
res_asymptotic <- mcnemar.test(table_2x2, correct = FALSE)
cat("\nAsymptotic McNemar (correct=FALSE):\n")
print(res_asymptotic)
cat(sprintf("  Manual check: (b-c)^2/(b+c) = (%d-%d)^2/(%d+%d) = %d/%d = %.4f\n",
            b, c, b, c, (b - c)^2, b + c, (b - c)^2 / (b + c)))

# ── Exact McNemar: base R uses exact binomial when correct=TRUE and ties present ──
# For clarity, compute directly with binom.test
res_exact_binom <- binom.test(min(b, c), b + c, p = 0.5)
cat(sprintf("\nExact McNemar (via binom.test): p = %.4f\n", res_exact_binom$p.value))

# ── Mid-p McNemar: Fagerland et al. (2013) — requires exact2x2 package ──
# Install once: install.packages("exact2x2")
# library(exact2x2)
# res_midp <- mcnemar.exact(table_2x2, midp = TRUE)
# cat(sprintf("Mid-p McNemar: p = %.4f\n", res_midp$p.value))
cat("\nFor mid-p correction (recommended when b+c < 25), use:\n")
cat("  library(exact2x2); mcnemar.exact(table_2x2, midp = TRUE)\n")
cat("  [exact2x2 package — Fagerland et al. 2013]\n")

# ── Interpretation notes ──
cat("\nKey reminders:\n")
cat("  - Concordant pairs (a, d) are IGNORED by the test.\n")
cat("  - McNemar tests marginal homogeneity; kappa tests agreement.\n")
cat("  - For multivariable adjustment after 1:1 matching:\n")
cat("    library(survival); clogit(outcome ~ exposure + strata(pair_id))\n")

/* ── Create the paired 2x2 frequency dataset ── */
/* Each record is one patient; pre and post are binary (1=Yes, 0=No) */
/* Reconstruct counts: a=10, b=15, c=5, d=10 */
data work.paired_binary;
  input pre_hosp post_hosp count;
  datalines;
1 1 10
1 0 15
0 1  5
0 0 10
;
run;

/* ── McNemar's test via PROC FREQ AGREE option ── */
/* WEIGHT count: each row represents `count` patients                   */
/* AGREE: prints McNemar's chi-square statistic, p-value, and kappa     */
/* TEST MCNEM: explicitly requests the McNemar statistic in the output  */
proc freq data=work.paired_binary;
  weight count;
  tables pre_hosp * post_hosp / agree norow nocol nopct;
  test mcnem;
  title "McNemar test: hospitalization pre-diagnosis vs post-diagnosis";
run;
/* Output includes:
   - McNemar's Chi-Square = (b-c)^2/(b+c) = (15-5)^2/20 = 5.0000, p approx 0.0253
   - Kappa: IGNORE for testing marginal change (kappa measures agreement, not change)
   - Simple Kappa Coefficient is a DIFFERENT quantity from McNemar                  */

/* ── Marginal proportions: compute manually from PROC MEANS ── */
/* Expand the frequency data to one row per patient */
data work.expanded;
  set work.paired_binary;
  do i = 1 to count;
    output;
  end;
  drop i;
run;
proc means data=work.expanded mean n;
  var pre_hosp post_hosp;
  title "Pre- and post-period hospitalization proportions";
run;
/* Expected output:
   pre_hosp  mean = 0.6250  (25/40)
   post_hosp mean = 0.3750  (15/40)
   Difference = 0.6250 - 0.3750 = 0.2500 (25 percentage points)            */

/* ── Note on exact and mid-p versions ── */
/* PROC FREQ AGREE uses the asymptotic chi-square version by default.
   For small discordant counts (b+c < 25), request the exact version:
     proc freq data=work.paired_binary;
       weight count;
       tables pre_hosp * post_hosp / agree;
       exact mcnem;  /* EXACT statement triggers exact binomial p-value   */
     run;
   Mid-p correction is not available natively in SAS PROC FREQ.
   For mid-p, export the table to R and use exact2x2::mcnemar.exact(midp=TRUE). */

Citations

1. [1]McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947;12(2):153-157.
2. [2]Fagerland MW, Lydersen S, Laake P. The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional. BMC Medical Research Methodology. 2013;13:91.