Test-Negative Design

In plain language

The test-negative design is a way to measure how well a vaccine works in the real world by studying only the people who already went to a clinic or hospital feeling sick and got tested for the disease in question. Everyone who tested positive for the target pathogen counts as a case; everyone who tested negative for it — but still showed up sick and got tested — counts as a control. Because both groups had to feel sick enough to seek care and to be tested, the design automatically cancels out most of the distortion that comes from vaccinated people simply being more likely to visit a doctor in the first place. The result is an odds ratio comparing vaccination rates in the two groups, and vaccine effectiveness is calculated as VE = (1 − odds ratio) × 100%.

The test-negative design (TND) estimates vaccine effectiveness (VE) — or, more generally, the effectiveness of a preventive or therapeutic intervention against a pathogen-specific outcome — by enrolling only people who present to a care setting with a defined clinical syndrome (e.g., acute respiratory illness, influenza-like illness) and who are then tested for the target pathogen. Those who test positive for the pathogen are the cases; those who test negative (the same syndrome, a different cause) are the controls. Vaccination status is ascertained for both groups and the effectiveness estimate is VE = 1 − OR, where the odds ratio compares the odds of vaccination among test-positives to the odds among test-negatives, typically from a logistic model adjusting for age, calendar time, and comorbidity. The single structural idea is that both cases and controls have already cleared the same care-seeking filter: they all felt sick enough to seek care and to be tested. If vaccinated and unvaccinated people differ in how readily they seek care (the "healthy-vaccinee" / healthcare-seeking confounder), that difference is largely conditioned out by restricting the study to people who sought care, because it acts on the probability of being sampled rather than on the pathogen-specific outcome itself.

Core conceptual distinction

The TND is not a cohort design and it is not a generic case-control study. (1) Versus a conventional case-control study, the controls are not population or hospital controls chosen for some unrelated reason; they are syndrome-matched, test-negative patients drawn from the very same testing stream as the cases. That common origin is what buys the control of differential care-seeking. (2) Versus a cohort VE study, the TND never enumerates a denominator of person-time at risk; it samples on presentation-and-test, so it estimates an odds ratio that approximates the rate ratio only under the design's identifying assumptions. (3) The estimand is pathogen-specific effectiveness against medically attended, tested disease — it says nothing about effectiveness against asymptomatic infection, transmission, or disease that never reaches a testing setting. Foppa's "case test-negative" formulation makes the sampling explicit and shows the conditions under which the OR identifies the rate ratio.

Validity rests on a small set of assumptions, and each maps to a bias

(a) Test accuracy: imperfect test sensitivity and specificity bias VE toward the null because some true cases land in the test-negative (control) group and vice versa; with a highly specific but imperfectly sensitive RT-PCR, modest sensitivity loss is usually non-differential and biases toward 0, but differential test performance by vaccination status (e.g., vaccinated infections having lower viral load and more false negatives) biases VE upward. (b) Equal care-seeking for vaccinated and unvaccinated, conditional on disease — if vaccination changes the probability of seeking care given the same severity, the conditioning is incomplete and residual selection remains. (c) No off-target protection: the vaccine must not also reduce the non-target (test-negative) illnesses, or those illnesses become an invalid control series and VE is biased (typically upward). (d) Confounding by indication / calendar time: vaccine uptake and pathogen circulation both vary over the season, so calendar time must be controlled, and confounding by underlying risk (frailty, comorbidity) is reduced but not eliminated by the design.

Pros, cons, and trade-offs

- vs the conventional `case-control` design: The TND's syndrome-matched, same-stream test-negative controls remove most differential healthcare-seeking and access bias that contaminate population/hospital controls; it is cheaper because cases and controls are captured passively from routine testing. Cost: the control series is only valid if the vaccine has no effect on the test-negative causes and the test is reasonably specific. Prefer the TND for routine, in-season VE surveillance against a pathogen-specific, laboratory-confirmable outcome; prefer a classic case-control when no clean syndrome-matched test-negative series exists or off-target effects are plausible. - vs a `cohort-retrospective` VE study: A cohort estimates a rate/risk ratio with an explicit denominator and can study multiple outcomes, but it must measure and adjust the full healthy-vaccinee confounding structure directly and is sensitive to outcome misclassification across the whole population. The TND sidesteps the care-seeking confounder by design but yields only an OR for medically attended tested disease. Prefer the cohort when person-time, multiple endpoints, or waning over long horizons is the question; prefer the TND when the dominant threat is care-seeking confounding and a specific laboratory endpoint is available. - vs `screening-method` / administrative VE: The Farrington screening method needs only case vaccination coverage and population coverage, but inherits all population-coverage measurement error and care-seeking bias. The TND is more robust to care-seeking but needs individual test data.

When to use

In-season influenza, COVID-19, rotavirus, pneumococcal, and dengue vaccine effectiveness from sentinel testing networks, emergency-department or hospital testing streams, and large EHR/claims-linked laboratory data; whenever a specific, laboratory-confirmable pathogen outcome exists, a syndrome-matched test-negative control series is naturally available, and differential healthcare-seeking between vaccinated and unvaccinated people is the chief threat to a cohort estimate. It is the default real-world VE design for the WHO and CDC sentinel platforms for exactly these reasons.

When NOT to use — and when it is actively misleading or dangerous

- When the vaccine has off-target effects on the test-negative illnesses. If the intervention also lowers the non-target causes of the syndrome, the test-negatives are depleted among the vaccinated and VE is biased upward — the single most dangerous TND misuse, because the bias looks like efficacy. - When the test is non-specific or differentially performing. Poor specificity dilutes VE toward the null; differential sensitivity by vaccination status (lower viral load in breakthrough infections) inflates VE. Do not run a TND on a syndromic or antigen test of unknown specificity and report the OR as if it were unbiased. - When vaccination changes care-seeking given disease. If vaccinated people who get infected are systematically less (or more) likely to present and be tested at the same severity, conditioning on testing does not remove the selection and VE is biased; the design's central assumption fails silently. - As an estimate of effectiveness against infection or transmission. The TND estimand is medically attended, tested disease only. Narrating a TND VE as protection against all infection overstates what was measured. - When test-negatives are not drawn from the same care/testing stream as cases. Pulling controls from a different setting reintroduces exactly the access/care-seeking confounding the design exists to remove.

Data-source operational depth

- Sentinel / surveillance testing networks (primary): The canonical substrate — a defined syndrome definition triggers a standardized test, and both arms come from one stream. Capture the test type, its specificity, the syndrome case definition, and calendar week, and adjust for week and site as fixed effects. - EHR-linked laboratory data: Test results and diagnoses are encounter-driven; differential leakage (testing done out-of-system) and informative testing (clinicians test sicker or higher-risk patients) can break the equal-care-seeking assumption. Restrict to a stable in-system population, define the syndrome from structured diagnosis plus an actual test order, and adjust for comorbidity and prior-year utilization. - Claims (FFS vs MA): Vaccination is captured from procedure/NDC/HCPCS codes and the outcome from a diagnosis paired with a test claim; Medicare Advantage enrollees generate no fee-for-service claims, so vaccination and testing can be differentially unobserved — restrict to FFS-observable person-time or VE is biased by ascertainment, not by the vaccine. - Hospital / ED test streams: Severity-based testing thresholds vary by site and over the season; include site and calendar-time terms and check that the test-negative case mix is stable across vaccination strata.

Worked example

Scenario

Imagine a sentinel flu surveillance network during a single influenza season. Every patient who walks into a participating clinic with fever plus cough or sore throat gets a rapid PCR test for influenza. Over the season, 340 patients meet that syndrome definition and receive a test. The analyst wants to estimate influenza vaccine effectiveness by comparing vaccination rates between the 140 patients who tested positive (cases) and the 200 patients who tested negative (controls). Because every patient in both groups had to feel sick enough to seek care and had to receive a test, the two groups share the same care-seeking history — that shared filter is what makes the comparison fair.

Dataset

One row per tested patient in a sentinel clinic registry. Each row records whether the patient was vaccinated this season and whether the influenza PCR test came back positive or negative.

Steps

• Lay out the 2×2 table. The columns are the two test-result groups (test-positive cases vs. test-negative controls) and the rows are vaccination status. From the 340 tested patients: 40 were vaccinated and test-positive (cell a), 100 were vaccinated and test-negative (cell b), 100 were unvaccinated and test-positive (cell c), and 100 were unvaccinated and test-negative (cell d). Totals: 140 cases (40 + 100), 200 controls (100 + 100).

• The 2×2 table in full: | | Test-positive (cases) | Test-negative (controls) | | Vaccinated | a = 40 | b = 100 | | Unvaccinated | c = 100 | d = 100 |

• Compute the odds ratio. The OR compares the odds of being vaccinated among cases to the odds of being vaccinated among controls: OR = (a × d) / (b × c) = (40 × 100) / (100 × 100) = 4,000 / 10,000 = 0.40.

• Interpret the OR. An OR of 0.40 means vaccinated patients had only 40% of the odds of being a flu case compared with unvaccinated patients who sought care and were tested — a large protective association.

• Convert the OR to vaccine effectiveness. VE = (1 − OR) × 100% = (1 − 0.40) × 100% = 0.60 × 100% = 60%. Vaccinated patients had 60% lower odds of testing positive for influenza.

• Why does restricting to tested patients help? Both vaccinated and unvaccinated people in this study had to feel sick enough to go to a clinic and receive a test. Any tendency for vaccinated people to visit the doctor more readily applies equally to both groups, so that tendency cancels out of the OR comparison rather than inflating the apparent effectiveness of the vaccine.

Result

OR = (40 × 100) / (100 × 100) = 4,000 / 10,000 = 0.40. VE = (1 − 0.40) × 100% = 60%. Among the 340 patients who were tested at sentinel clinics this season, vaccinated individuals had 60% lower odds of a positive influenza test than unvaccinated individuals, after accounting for the shared care-seeking filter.

Runnable example

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf

# Illustrative sentinel-testing data: each row is one syndrome-presenting, tested patient.
rng = np.random.default_rng(7)
n = 4000
week = rng.integers(40, 53, n)                       # epidemiologic week
vaccinated = rng.binomial(1, 0.55, n)                # vaccination status
age = rng.normal(50, 18, n).clip(1, 95)
# True VE ~ 60%: vaccination lowers the odds of being a (test-positive) case.
lin = -0.4 + np.log(0.40) * vaccinated + 0.01 * (age - 50) + 0.02 * (week - 46)
test_positive = rng.binomial(1, 1 / (1 + np.exp(-lin)))   # 1 = pathogen+, 0 = pathogen-
df = pd.DataFrame(dict(test_positive=test_positive, vaccinated=vaccinated,
                       age=age, week=week))

# Cases = test-positive, controls = test-negative; model the odds of being a case.
m = smf.logit("test_positive ~ vaccinated + age + C(week)", data=df).fit(disp=0)
aor = np.exp(m.params["vaccinated"])
ci_lo, ci_hi = np.exp(m.conf_int().loc["vaccinated"])
ve     = 1 - aor                                     # VE = 1 - adjusted OR
ve_lo  = 1 - ci_hi                                   # CI flips because VE = 1 - OR
ve_hi  = 1 - ci_lo
print(f"adjusted OR = {aor:.3f}")
print(f"VE = {ve*100:.1f}%  (95% CI {ve_lo*100:.1f}% to {ve_hi*100:.1f}%)")

set.seed(7)
n     <- 4000
week  <- sample(40:52, n, replace = TRUE)            # epidemiologic week
vacc  <- rbinom(n, 1, 0.55)                          # vaccination status
age   <- pmin(pmax(rnorm(n, 50, 18), 1), 95)
lin   <- -0.4 + log(0.40) * vacc + 0.01 * (age - 50) + 0.02 * (week - 46)
tpos  <- rbinom(n, 1, plogis(lin))                   # 1 = test-positive (case), 0 = control
dat   <- data.frame(test_positive = tpos, vaccinated = vacc,
                    age = age, week = factor(week))

# Cases = test-positive, controls = test-negative; logistic model for being a case.
fit  <- glm(test_positive ~ vaccinated + age + week, family = binomial, data = dat)
beta <- coef(summary(fit))["vaccinated", ]
aor  <- exp(beta["Estimate"])
ci   <- exp(beta["Estimate"] + c(-1, 1) * 1.96 * beta["Std. Error"])
ve    <- 1 - aor                                     # VE = 1 - adjusted OR
ve_ci <- 1 - rev(ci)                                 # flip limits: VE = 1 - OR
cat(sprintf("adjusted OR = %.3f\n", aor))
cat(sprintf("VE = %.1f%% (95%% CI %.1f%% to %.1f%%)\n",
            100 * ve, 100 * ve_ci[1], 100 * ve_ci[2]))