concept

Fisher's Exact Test

A significance test for 2×2 (and larger) contingency tables that computes the p-value by summing hypergeometric probabilities over all table arrangements at least as extreme as the observed one — conditioning on the row and column totals — making it exact and valid even when expected cell counts are very small or zero; in RWE its principal domain is the sparse safety table (rare adverse events, small exposed cohorts) where the large-sample chi-square approximation breaks down, though the test's conservatism under full conditioning means unconditional exact tests or the mid-p correction are often the superior choice.

Inferential_Statisticsstatisticsprimitivehypothesis-testingcategorical-datarare-eventsexact-testcontingency-tablesafety-surveillance

Methods reference only. Use primary source citations and local policy before applying this in a study protocol, regulatory submission, payer dossier, or clinical decision.

In plain language

Fisher's Exact Test is a way to ask whether two groups differ in how often an event occurs, specifically designed for situations where the numbers are too small to trust the usual chi-square shortcut. It works by counting every possible way the data could have been arranged if there were truly no difference, and then figuring out how rare the actual result is by exact arithmetic on those counts — no approximations needed. In real-world evidence studies it most often appears in safety tables for rare adverse events, where one or more cells in the table may contain zero or single-digit counts. A key limitation: because it is deliberately cautious, it misses real effects more often than necessary, so researchers sometimes use a small correction called mid-p to get closer to the right answer without losing the exactness guarantee.

What Fisher's Exact Test does and why "exact" matters

Fisher's Exact Test answers one question: given that we observe a 2×2 table with fixed marginal totals, how often would we see this arrangement — or one even more lopsided — if there were truly no association between the row and column variables?

The test works by conditioning on the marginal totals (both row totals and both column totals are treated as fixed), which reduces the problem to a single hypergeometric distribution. Under the null hypothesis of no association, the upper-left cell count follows a hypergeometric distribution with parameters determined by the four marginals. The exact p-value is the sum of hypergeometric probabilities for all tables with those same marginals that are at least as extreme (have upper-left cell count as small or smaller) as the observed table.

Because the p-value is computed directly from the hypergeometric PMF rather than from a chi-square approximation, it is "exact" in the sense that it does not depend on an asymptotic argument. The type-I error rate is guaranteed not to exceed the nominal alpha for any table size — including tables with n = 10 patients or with one or more cells containing zeros. This guarantee is precisely what Fisher's test offers that the Pearson chi-square test cannot.

The hypergeometric probability in detail

A 2×2 table has four cells denoted as follows: a (row 1 / col 1), b (row 1 / col 2), c (row 2 / col 1), d (row 2 / col 2). The row totals are R1 = a + b and R2 = c + d; the column totals are C1 = a + c and C2 = b + d; the grand total is N = R1 + R2.

Under the null hypothesis of no association, with all margins fixed, the probability of observing exactly a events in the upper-left cell is the hypergeometric PMF:

P(X = a) = C(R1, a) × C(R2, C1 - a) / C(N, C1)

where C(n, k) = n! / [k! × (n-k)!] is the binomial coefficient. The p-value for a one-sided test (testing whether the upper-left cell count is smaller than expected) is the sum of this probability over all values of a from 0 up to the observed value; the standard two-sided p-value doubles this sum (or, more precisely, sums over all tables whose hypergeometric probability does not exceed that of the observed table).

Why conditioning on margins is controversial

Ronald Fisher's original justification for conditioning on both margins rested on the "Lady Tasting Tea" experiment, where both the row and column totals are genuinely fixed by design. In most medical and epidemiological studies this is not the case: only the row totals (sample sizes per group) are fixed, not the column totals (event counts). When only one margin is fixed, conditioning on the other margin discards information about the population proportion of events — it treats a quantity that could distinguish the null from the alternative as nuisance.

This has a practical consequence: Fisher's test is conservative — its actual type-I error rate is strictly below the nominal alpha for most discrete tables, because the hypergeometric distribution is discrete and the exact tail probability rarely hits alpha exactly. Conservatism means lower power than you could have if you used an unconditional exact test (Barnard's test) or the mid-p correction.

The conservatism critique and the mid-p alternative

The conservatism of Fisher's test is well-established in the statistical literature. Lydersen, Fagerland, and Laake (2009) conducted a comprehensive simulation comparing eighteen tests for 2×2 tables and reached the conclusion that Fisher's test is often not the best choice: its actual size is far below the nominal level in many scenarios (sometimes less than half of alpha), leading to a systematic loss of power that is particularly damaging in small studies where power is already scarce.

The mid-p correction addresses this by computing:

mid-p = (1/2) × P(X = a_observed) + P(X < a_observed)

This is not a valid frequentist p-value in the strict sense (it does not guarantee conservative type-I error control), but it brings the actual size much closer to the nominal level while remaining more conservative than the chi-square approximation. Lydersen et al. (2009) recommend the mid-p correction as a practical improvement over the standard Fisher p-value for most small-sample 2×2 tables. Unconditional exact tests (Barnard's test, Boschloo's test) are theoretically superior in terms of power, but computationally heavier and not universally implemented.

The zero-cell problem and its consequences for odds ratios

In sparse safety tables — a rare adverse event occurring in 0 of 10 exposed patients versus 0 of 50 controls, or 1 of 100 versus 0 of 200 — the conditional odds ratio (OR) is undefined: zero cells make the cross-product ratio 0/0 or a/0. Fisher's test can still produce a valid p-value in these situations because the hypergeometric calculation does not require the OR to be defined, but it cannot produce a finite OR or OR-based confidence interval.

Three common remedies for the zero-cell OR problem:

1. Continuity correction (add 0.5 to each cell): produces a finite OR but the choice of 0.5 is arbitrary and the resulting CI is not exact. 2. Exact conditional CI (computed from the hypergeometric distribution): available from fisher.test in R and PROC FREQ EXACT FISHER in SAS; this CI is exact in the conditional sense but inherits Fisher's conservatism. 3. Firth penalized logistic regression: the modern recommended approach for sparse or zero-cell tables when an adjusted OR is needed. Firth's method adds a Jeffreys-prior penalty to the log-likelihood, which removes the separation problem and produces finite point estimates and profile likelihood CIs without any ad-hoc correction. When the analysis must adjust for covariates (as is nearly always the case in observational RWE), exact logistic regression or Firth logistic regression is the appropriate next step.

RWE and safety surveillance applications

Fisher's Exact Test appears in RWE most often in three contexts:

Spontaneous pharmacovigilance and signal tables: Disproportionality tables in regulatory safety databases often involve rare suspected adverse drug reactions; the MedDRA preferred term × drug pairing table may have single-digit cell counts even in large spontaneous reporting databases. Fisher's test is one of several signal-detection metrics, alongside the reporting odds ratio and proportional reporting ratio.

Small clinical cohorts and registry subgroups: A pragmatic trial or registry study in a rare disease may enroll 20 to 50 patients per arm. Serious adverse events in these cohorts produce 2×2 tables with expected counts below 5, violating the chi-square approximation. Fisher's test or the mid-p correction is appropriate here, with the caveat that power is limited and the absence of a statistically significant finding cannot be interpreted as proof of safety.

Post-market pharmacoepidemiology safety tables: Early post-approval surveillance of a newly approved drug may cover only a few thousand exposed patients. If the event of interest is rare (background rate < 1 per 1,000), observed counts may be 0, 1, or 2 even in large studies. Fisher's test handles these counts; TreeScan and maxSPRT add sequential analysis for repeated looks.

Scale point: when Fisher's test is computationally unnecessary

In large administrative claims databases — 100,000 to 10,000,000 person-years of follow-up — expected cell counts for even moderately rare events (incidence rate 0.1 per 100) will be in the hundreds or thousands per cell. At that scale the chi-square approximation is excellent, Fisher's test offers no advantage, and the computational cost (combinatorial for large N) is unnecessary. Fisher's test is the right tool when expected counts are small, not when the database is large. An analyst working in Medicare claims with 500,000 exposed patients comparing a cardiovascular event rate of 5% does not need Fisher's test.

Pros, cons, and trade-offs

Pros: - Exact: the p-value is guaranteed to not exceed alpha under the null regardless of table size; valid for any n, including n = 5 or n = 0 in a cell. - Universally implemented: scipy.stats.fisher_exact, R's fisher.test, SAS PROC FREQ EXACT FISHER are available in all analytical pipelines without additional packages. - Handles zero cells: produces a valid p-value even when one cell contains zero events, unlike chi-square which returns NaN or requires continuity correction. - Provides exact conditional CI for the OR: fisher.test in R returns the conditional maximum-likelihood OR estimate and an exact CI alongside the p-value.

Cons: - Conservative: actual type-I error is below nominal alpha due to discreteness; power is lower than unconditional tests or mid-p correction in the same scenarios. - Does not directly estimate an effect size: the test produces a p-value; the conditional OR and CI are a separate output and are undefined when cells contain zeros. - Conditions on both margins: theoretically appropriate only when both margins are fixed by design; for observational 2×2 tables this assumption is debatable and leads to the conservatism described above. - Computationally expensive for large N: the exact calculation becomes slow for tables with very large marginal totals, though this is rarely a practical problem because Fisher's test is only needed when N is small. - Cannot adjust for confounders: like chi-square, Fisher's test produces an unadjusted association measure; when confounders are present, route to exact logistic regression or Firth logistic regression.

When to use

Use Fisher's Exact Test when:

Any expected cell count in a 2×2 (or larger) contingency table is below 5; the
One or more observed cells contain zero events; Fisher's test computes a valid p-value
The study involves a rare adverse event in a small cohort (n < 50 per group); adverse
You need a conservative test where type-I error is guaranteed not to exceed alpha,
As the "exact" comparator alongside chi-square in a sensitivity analysis to confirm
In descriptive pharmacovigilance tables where a p-value is requested by convention,

Consider the mid-p correction when you use Fisher's test in a setting where power matters and you accept that the actual type-I error rate may approach (but not exceed) the nominal alpha more closely than the conservative Fisher p-value allows.

When NOT to use

Do not use Fisher's Exact Test when:

All expected cell counts are ≥ 5 and n per group is adequate: chi-square is the
The design is paired: a 2×2 table comparing pre-post classifications, matched pairs,
An adjusted odds ratio is needed: Fisher's test is an unadjusted marginal test; for
The table is larger than 2×2 with ordered categories: ordinal × ordinal associations
The study is large (n >> 1,000 per group) with common events: at this scale chi-square
A relative risk or rate difference is the target estimand: Fisher's test is built

Implementation note across languages

All three implementations below show the same 2×2 safety table (3 adverse events in 10 exposed vs 0 in 10 unexposed), compute the exact p-value, report the conditional OR and CI, and apply the mid-p correction by hand. This mirrors the worked example in the beginner layer so the analyst can trace from manual arithmetic to production code.

Interpreting the output

In the worked example, 3 of 10 patients in the drug arm experienced a rash and 0 of 10 in the placebo arm did. The one-sided exact p-value — the hypergeometric probability of observing 3 or more events in the exposed arm given fixed marginals of 10 exposed, 10 unexposed, and 3 total events — is 2/19 (approximately 0.105). The two-sided p-value is 4/19 (approximately 0.211). The mid-p one-sided value, which halves the probability of the observed table before accumulating the tail, is approximately 0.053.

(1) Formal interpretation. Under the null hypothesis of no association and conditioning on both sets of marginal totals, the exact probability of an arrangement as extreme or more extreme than the one observed is approximately 0.11 (one-sided) or 0.21 (two-sided). Neither exceeds the conventional alpha = 0.05 threshold, and the test does not reject the null. The mid-p correction brings the one-sided value to approximately 0.053 — still marginally above 0.05. Fisher's conservatism (which arises from conditioning on both margins) makes it the most cautious of the available exact tests; the non-rejection does not rule out a real adverse-event signal — it reflects the severe limitation of the sample size.

(2) Practical interpretation. Three rash events in 10 exposed patients versus zero in 10 unexposed patients is a directionally concerning imbalance that does not achieve conventional statistical significance in a study of 20 total patients. Fisher's exact test is the appropriate choice here because expected cell counts are well below 5, making the chi-square approximation unreliable. A non-significant result in a study this small cannot be interpreted as evidence of no rash risk — the study simply lacks the power to detect or rule out signals of this magnitude. Safety surveillance for rare adverse events requires much larger exposed populations and sequential monitoring methods before a signal can be confirmed or excluded.

Worked example

Scenario

A safety monitor is reviewing a phase II trial in a rare disease. Ten patients received the new drug and ten received placebo. Three patients in the drug arm experienced a serious rash; zero patients in the placebo arm did. The expected counts are too small for chi-square (expected events in placebo cell = 1.5), so the monitor computes Fisher's Exact Test by hand to decide whether the imbalance is unlikely under the null hypothesis of no drug effect on rash.

Dataset

2x2 adverse-event table. Rows = treatment arm; columns = rash (yes/no). Row totals and column totals (margins) are used as the fixed inputs to Fisher's calculation.

arm	rash_yes	rash_no	row_total
Drug	3	7	10
Placebo		10	10
col_total	3	17	20

Steps

Result

C(20,3) = 1140. P(X=0) = 120/1140, P(X=1) = 450/1140, P(X=2) = 450/1140, P(X=3) = 120/1140. Sum = 1140/1140 = 1. One-sided Fisher p-value = P(X=3) = 120/1140 = 2/19 which equals approximately 0.1053. Two-sided p = 240/1140 = 4/19 which equals approximately 0.2105. Mid-p one-sided = 60/1140 = 1/19 which equals approximately 0.0526. With only 10 patients per arm the test has very low power; neither the exact p-value nor the mid-p reaches conventional significance after two-sided correction, illustrating why rare-event detection in small cohorts requires sequential designs or pooled post-market data rather than a single-study test.

Runnable example

python implementation

Fisher's Exact Test using scipy.stats.fisher_exact. Shows the 2x2 safety table from the worked example (3/10 vs 0/10 adverse events), computes the two-sided exact p-value, extracts the conditional odds ratio, and manually computes the mid-p correction from...

from scipy import stats

# ── Safety table: drug vs placebo, rash yes/no ──────────────────────────────────
# Rows: [Drug, Placebo]; Columns: [Rash, No_Rash]
table = [[3, 7],   # Drug:    3 rash, 7 no rash
         [0, 10]]  # Placebo: 0 rash, 10 no rash

# ── 1. Standard two-sided Fisher exact test ──────────────────────────────────────
odds_ratio, p_two_sided = stats.fisher_exact(table, alternative="two-sided")
print(f"Odds ratio (cross-product):   {odds_ratio:.3f}")
print(f"Two-sided exact p-value:       {p_two_sided:.4f}")

# One-sided: testing whether drug arm has HIGHER rash rate
_, p_one_sided = stats.fisher_exact(table, alternative="greater")
print(f"One-sided exact p-value:       {p_one_sided:.4f}")

# ── 2. Mid-p correction (manual, from hypergeometric PMF) ─────────────────────────
# Under the null: X ~ Hypergeometric(N=20, K=3 rash events total, n=10 drug arm)
# scipy.stats.hypergeom(M, n, N): M=population, n=success states, N=draws
N_total = 20   # grand total
K_events = 3   # total rash events (column 1 total)
n_drug = 10    # drug arm size (row 1 total)
a_obs = 3      # observed drug-arm rash count

rv = stats.hypergeom(N_total, K_events, n_drug)
p_at_obs = rv.pmf(a_obs)          # P(X = 3)
p_more_extreme = rv.sf(a_obs)     # P(X > 3) = 0 (max is 3)
mid_p_one_sided = 0.5 * p_at_obs + p_more_extreme
print(f"\nMid-p one-sided:               {mid_p_one_sided:.4f}")
print(f"P(X=3) from hypergeom PMF:     {p_at_obs:.4f}")

# ── 3. Hypergeometric PMF for all possible a values ──────────────────────────────
print("\nHypergeometric probability table:")
for a in range(0, K_events + 1):
    p = rv.pmf(a)
    print(f"  P(X={a}) = C(10,{a}) x C(10,{K_events-a}) / C(20,{K_events}) = {p:.6f}")

# ── 4. Scale check: chi-square on a larger table ──────────────────────────────────
# In claims with 500 drug / 500 placebo and 5% event rate, chi-square is fine
large_table = [[25, 475], [15, 485]]  # 5% vs 3% events, n=500 per arm
chi2, chi2_p, dof, expected = stats.chi2_contingency(large_table, correction=False)
_, fisher_p_large = stats.fisher_exact(large_table)
print(f"\nLarge-sample (n=500/arm):")
print(f"  Expected min cell:  {expected.min():.1f}  (>> 5: chi-square valid)")
print(f"  Chi-square p:       {chi2_p:.4f}")
print(f"  Fisher exact p:     {fisher_p_large:.4f}  (nearly identical; chi-square sufficient)")

r implementation

Fisher's Exact Test in base R. Demonstrates fisher.test with both two-sided and one-sided alternatives, extracts the conditional maximum-likelihood OR and exact confidence interval, computes mid-p manually from dhyper, and shows the scale-point comparison...

# ── Safety table: drug vs placebo, rash yes/no ──────────────────────────────────
tab <- matrix(c(3, 0, 7, 10), nrow = 2,
              dimnames = list(Arm = c("Drug", "Placebo"),
                              Rash = c("Yes", "No")))
cat("Observed table:\n"); print(tab)
cat("Expected counts (under null):\n")
print(chisq.test(tab, correct = FALSE)$expected)

# ── 1. Two-sided Fisher exact (standard clinical report) ──────────────────────────
ft2 <- fisher.test(tab, alternative = "two.sided")
cat("\nTwo-sided Fisher exact test:\n")
print(ft2)
# NOTE: ft2$estimate is the conditional MLE OR (differs from cross-product at small n)
# ft2$conf.int is the exact conditional 95% CI for this OR

# ── 2. One-sided (testing drug > placebo for rash) ───────────────────────────────
ft1 <- fisher.test(tab, alternative = "greater")
cat("\nOne-sided Fisher exact (drug > placebo):\n")
cat(sprintf("  p-value: %.4f\n", ft1$p.value))

# ── 3. Mid-p correction (manual from dhyper) ─────────────────────────────────────
# X ~ Hypergeometric(m=3 total rash, n=17 total no-rash, k=10 drug arm)
# dhyper(x, m, n, k): P(X=x) where m = K_events, n = N-K_events, k = n_drug
m_events  <- 3   # total rash events = column 1 total
n_no_event <- 17  # total no-rash = column 2 total
k_drug    <- 10  # drug arm size
a_obs     <- 3   # observed drug-arm rash count

p_at_obs    <- dhyper(a_obs, m_events, n_no_event, k_drug)
p_more_extreme <- phyper(a_obs, m_events, n_no_event, k_drug, lower.tail = FALSE)
mid_p_one_sided <- 0.5 * p_at_obs + p_more_extreme
cat(sprintf("\nMid-p one-sided: %.4f (vs conservative Fisher one-sided: %.4f)\n",
            mid_p_one_sided, ft1$p.value))

# ── 4. PMF table for all possible a values ────────────────────────────────────────
cat("\nHypergeometric PMF for all a = 0..3:\n")
for (a in 0:3) {
  cat(sprintf("  P(X=%d) = %.6f\n", a, dhyper(a, m_events, n_no_event, k_drug)))
}

# ── 5. Scale point: large table where chi-square is adequate ─────────────────────
large_tab <- matrix(c(25, 15, 475, 485), nrow = 2)
exp_min <- min(chisq.test(large_tab, correct = FALSE)$expected)
chi_p   <- chisq.test(large_tab, correct = FALSE)$p.value
fish_p  <- fisher.test(large_tab)$p.value
cat(sprintf("\nLarge n=500/arm: expected min = %.1f; chi2 p = %.4f; Fisher p = %.4f\n",
            exp_min, chi_p, fish_p))
cat("All expected counts >> 5: chi-square is appropriate; Fisher adds nothing.\n")