Positive and Negative Predictive Value

In plain language

When you build a computer rule (an "algorithm") to flag patients with a disease in claims or EHR data, positive predictive value (PPV) asks: of the records the rule flagged as cases, what fraction are really cases? Negative predictive value (NPV) asks the mirror question: of the records it flagged as non-cases, what fraction are really non-cases? You compute them straight from a 2x2 table of the rule's guess versus the chart-confirmed truth: PPV = true positives / all flagged-positive, NPV = true negatives / all flagged-negative. The catch worth remembering: both numbers move with how common the disease is in the population you tested, so a PPV measured in one cohort can be very different in a rarer one even when the rule itself never changed.

Positive predictive value (PPV)

and negative predictive value (NPV) answer the question a downstream analyst actually faces: given the test (algorithm) result, what is the probability the truth matches it? From the 2x2 validation table with truth in columns — true positives (TP), false positives (FP), false negatives (FN), true negatives (TN) — PPV = TP / (TP + FP) is the probability of true disease given a positive result (P(D+|T+)), and NPV = TN / (TN + FN) is the probability of true non-disease given a negative result (P(D−|T−)). These condition on the rows (the test result) rather than the columns (the truth), which is the precise sense in which they are the inverse-conditional companions of sensitivity and specificity.

Core idea

Because PPV and NPV condition on the test result, they depend on how common the disease is in the tested population. Bayes' theorem makes the dependence explicit. Writing prevalence p = P(D+), sensitivity Se = P(T+|D+), and specificity Sp = P(T−|D−):

PPV = (Se p) / (Se p + (1 - Sp) (1 - p)) NPV = (Sp (1 - p)) / (Sp (1 - p) + (1 - Se) p)

Sensitivity and specificity are intrinsic to the test on a fixed case spectrum; PPV and NPV layer prevalence on top of them. The operationally crucial consequence: a highly sensitive, highly specific algorithm can still have a low PPV when the outcome is rare, because the small number of true cases is swamped by false positives drawn from the large pool of non-cases. This is exactly the regime of most safety outcomes and incident events in claims, where prevalence in the cohort is a few percent or less.

Why charting algorithm-positives yields PPV, not sensitivity

The most common RWE validation design pulls medical charts only for records the algorithm flagged positive and adjudicates them. That design observes TP and FP (the positives that are true vs false) and therefore identifies PPV = TP/(TP+FP) directly. It cannot observe FN — the true cases the algorithm missed are, by construction, not in the algorithm-positive sample — so sensitivity = TP/(TP+FN) is unidentifiable from algorithm-positive charts alone. To estimate sensitivity you must also chart a sample of algorithm-negatives (or link to an external high-coverage reference) so that FN become observable. Labeling a positive-predictive-value validation as a "sensitivity" estimate is the single most frequent error in claims-algorithm papers.

The Bayesian relation and its practical inversion

Because PPV/NPV mix the test's intrinsic accuracy with the population's prevalence, two moves recur. (1) Transport: a PPV estimated in a high-prevalence validation sample does not carry to a low-prevalence analysis cohort; recompute it from sensitivity, specificity, and the new prevalence via the Bayes formula, or better, design the validation in the target population. (2) Back-out: if you have PPV from algorithm-positives and an external prevalence and specificity, you can solve the Bayes relation for the implied sensitivity — useful but fragile, since it propagates the uncertainty of every input.

Pros, cons, and trade-offs

- vs sensitivity/specificity (`sensitivity-specificity-rwe`): PPV/NPV are what a downstream user needs to interpret a flag in a specific cohort — they answer "is this record true?" Their cost is prevalence-dependence: they do not transport across cohorts of differing case frequency and cannot, by themselves, characterize the test. Prefer PPV/NPV to interpret and weight flagged records in the analysis cohort; prefer sensitivity/specificity to characterize the algorithm and feed misclassification corrections. - vs likelihood ratios (`likelihood-ratios-diagnostic-rwe`): A likelihood ratio is prevalence-independent and updates pre-test odds to post-test odds at any prevalence, so it separates the test's evidentiary weight from the population. PPV is the resulting post-test probability at one specific prevalence. Prefer a likelihood ratio to summarize evidentiary strength portably; prefer PPV to state the concrete probability in the cohort at hand. - vs raw count of algorithm-positives: Using algorithm-positives as if every flag were a true case ignores the FP fraction; PPV makes the contamination explicit and lets you either restrict to high-PPV definitions or correct the downstream estimate. Always prefer the PPV-aware analysis over treating flags as truth.

When to use

Report PPV (with an exact binomial CI) whenever you validate an algorithm by sampling and adjudicating algorithm-positive records — the dominant, low-cost claims design; report PPV and NPV together when you have sampled both result strata; use the Bayes formula to project PPV/NPV into the target analysis cohort's prevalence; and quote PPV when the operational question is "what fraction of the patients my algorithm flags are real?" — e.g., to size a chart- abstraction workload or to weight cases in an outcome analysis.

When NOT to use — and when it is actively misleading or dangerous

- As a stand-in for sensitivity. A high PPV says little about how many true cases were missed. An algorithm tuned for PPV (specific, restrictive code rules) can have poor sensitivity, biasing incidence downward and attenuating effect estimates. Reporting only PPV and calling the algorithm "validated" hides the FN problem. - Transporting a PPV across prevalences. A PPV measured where the outcome is 20% prevalent overstates the PPV in a cohort where it is 2%. Quoting the validation-sample PPV in a different cohort without re-deriving from sensitivity, specificity, and the new prevalence is actively misleading. - In a rare-outcome cohort, reassuring with sensitivity/specificity alone. "95% sensitive and 95% specific" sounds definitive, but at 1% prevalence PPV is only ~16% — most flags are false. Presenting the operating characteristics without the implied PPV gives false confidence in the positives. - When the validation frame mis-measures prevalence. If MA-only person-time (no fee-for-service claims) inflates the apparent denominator, the prevalence feeding the Bayes calculation is wrong and the projected PPV/NPV are wrong.

Data-source operational depth

- Claims (FFS): PPV from algorithm-positive chart review is the standard, inexpensive deliverable; it is the right metric for "how many flags are real," but it inherits the cohort's prevalence and must be re-projected for any cohort with different case frequency. NPV is rarely estimated because charting algorithm-negatives is costly. Medicare Advantage enrollees produce no FFS claims, so MA-only spans appear as algorithm-negatives and distort the prevalence that drives PPV/NPV — restrict the validation frame to FFS-observable person-time. - EHR: Encounter-driven capture means an algorithm-negative may be truly positive but seen elsewhere ("leakage"), inflating apparent FN and depressing NPV; structured labs/problem lists and NLP raise PPV by sharpening the case definition. Report the prevalence in the validation sample so PPV can be transported. - Registry/linked: Registry linkage supplies the reference standard and a credible prevalence; linked claims-EHR- registry data lets PPV be both estimated and transported, and lets NPV be estimated by adjudicating algorithm-negatives against the registry.

Worked claims example

The same incident-HF claims algorithm validated for sensitivity/specificity is, in routine practice, validated more cheaply by charting only the records it flags. Of 300 adjudicated algorithm-positives, 261 are confirmed HF and 39 are not: PPV = 261 / (261 + 39) = 0.870 (87.0%), exact binomial 95% CI 0.827–0.906. If the team also charts 300 algorithm-negatives and finds 18 missed true cases (FN) against 282 confirmed non-cases (TN), NPV = 282 / (282 + 18) = 0.940 (94.0%). Crucially, these PPV/NPV depend on the prevalence in the validation frame. To apply the algorithm in a younger commercial cohort where incident HF is far rarer, the team does not reuse 87% PPV; instead it takes the prevalence-invariant sensitivity (93.5%) and specificity (87.9%) from the stratified validation and re-derives PPV by Bayes at the new prevalence. At prevalence p = 0.02, PPV = (0.935 0.02) / (0.935 0.02 + 0.121 * 0.98) = 0.136 — only 13.6% of flags would be true HF, an order-of-magnitude collapse from the 87% measured in the enriched validation sample, and a number that fundamentally changes whether the algorithm-positives can be used as outcomes without correction.

Interpreting the output

In the worked example, the validation sample yields PPV = 261/300 ≈ 0.870 (87.0%) in an enriched sample, but at a community prevalence of 2% the Bayes-projected PPV collapses to ≈ 0.136 (13.6%).

(1) Formal interpretation. PPV = TP / (TP + FP) = 0.870 is the conditional probability that a patient flagged positive by the algorithm is a true case, given the prevalence in the validation sample. NPV = TN / (TN + FN) ≈ 0.940 is the conditional probability that a flagged-negative is a true non-case, again at that prevalence. Unlike sensitivity and specificity, PPV and NPV are not fixed properties of the algorithm — they are derived from (sensitivity, specificity, and prevalence) via Bayes' theorem. Holding sensitivity at 0.935 and specificity at 0.879 constant, moving prevalence from roughly 50% in the enriched validation design to 2% in a community cohort drives PPV from 87% to 13.6%. This is not a failure of the algorithm; it is a mathematical consequence of the low signal-to-noise ratio at low prevalence.

(2) Practical interpretation. Never transport a PPV or NPV estimate from a validation study to a different population without checking that prevalences are comparable. When using algorithm-flagged cases as outcomes in an effectiveness study, a PPV of 13.6% at 2% prevalence means the outcome is contaminated by roughly 86% false positives — severe enough to require either a sensitivity analysis excluding unvalidated events or a matrix misclassification correction using the prevalence-invariant (sensitivity, specificity) pair. Report both the observed PPV and the Bayes-projected PPV at the target prevalence.

Worked example

Scenario

A team has a computer rule that flags incident heart-failure (HF) cases in claims data. To check it, they pull medical charts for 300 records the rule flagged as cases and 300 it flagged as non-cases, then a clinician adjudicates each one as truly HF or not. The 2x2 table below counts how the rule's guess (rows) lined up with the chart truth (columns). From it we compute PPV and NPV, then show why those exact numbers do not carry over to a cohort where HF is much rarer.

Dataset

The adjudicated 2x2 confusion table an analyst builds: the algorithm's call (rows) crossed with chart-confirmed truth (columns). Each cell is a patient count.

Steps

• Read the cells off the table: true positives TP = 261, false positives FP = 39, false negatives FN = 18, true negatives TN = 282.

• PPV conditions on the algorithm-positive row (the 300 records it flagged as cases): PPV = TP / (TP + FP) = 261 / (261 + 39) = 261 / 300.

• NPV conditions on the algorithm-negative row (the 300 it flagged as non-cases): NPV = TN / (TN + FN) = 282 / (282 + 18) = 282 / 300.

• Now see why these numbers depend on prevalence. The algorithm's intrinsic accuracy here is sensitivity Se = 0.935 and specificity Sp = 0.879 (these do not change with how common HF is). Feed them through Bayes' theorem: PPV = (Sep) / (Sep + (1-Sp)*(1-p)).

• In a moderate-prevalence cohort where HF is 20% common (p = 0.20): PPV = (0.9350.20) / (0.9350.20 + 0.121*0.80) = 0.187 / 0.2838 = 0.659 -- about 66% of flags are real.

• In a young commercial cohort where incident HF is rare, 2% (p = 0.02): same Se and Sp, but PPV = (0.9350.02) / (0.9350.02 + 0.121*0.98) = 0.0187 / 0.13732 = 0.136 -- only about 14% of flags are real. Nothing about the rule changed; only the prevalence did.

Result

PPV = 261 / 300 = 0.870 (87.0%) and NPV = 282 / 300 = 0.940 (94.0%) in this validation sample. But because PPV depends on prevalence, the same algorithm (Se = 0.935, Sp = 0.879) gives PPV = 0.659 at 20% prevalence and collapses to PPV = 0.136 at 2% prevalence -- so the 87% must never be reused in a rarer cohort without re-deriving it via Bayes.

Runnable example

from scipy.stats import beta

def exact_binom_ci(k: int, n: int, conf: float = 0.95):
    alpha = 1.0 - conf
    lo = 0.0 if k == 0 else beta.ppf(alpha / 2, k, n - k + 1)
    hi = 1.0 if k == n else beta.ppf(1 - alpha / 2, k + 1, n - k)
    return lo, hi

def ppv_npv(tp: int, fp: int, fn: int, tn: int) -> dict:
    # PPV conditions on the positive row (TP + FP); NPV on the negative row (TN + FN).
    ppv = tp / (tp + fp)
    npv = tn / (tn + fn)
    return {"ppv": ppv, "ppv_ci": exact_binom_ci(tp, tp + fp),
            "npv": npv, "npv_ci": exact_binom_ci(tn, tn + fn)}

def project_ppv_npv(sens: float, spec: float, prevalence: float) -> dict:
    # Bayes' theorem: PPV/NPV at a target prevalence from prevalence-invariant Se, Sp.
    p = prevalence
    ppv = (sens * p) / (sens * p + (1 - spec) * (1 - p))
    npv = (spec * (1 - p)) / (spec * (1 - p) + (1 - sens) * p)
    return {"ppv_at_prevalence": ppv, "npv_at_prevalence": npv}

# Worked claims example: PPV/NPV from validation, then projection to a rare-outcome cohort.
obs = ppv_npv(tp=261, fp=39, fn=18, tn=282)
print(f"PPV = {obs['ppv']:.3f}  95% CI {obs['ppv_ci'][0]:.3f}-{obs['ppv_ci'][1]:.3f}")
print(f"NPV = {obs['npv']:.3f}  95% CI {obs['npv_ci'][0]:.3f}-{obs['npv_ci'][1]:.3f}")
proj = project_ppv_npv(sens=0.935, spec=0.879, prevalence=0.02)
print(f"Projected PPV at 2% prevalence = {proj['ppv_at_prevalence']:.3f}")  # ~0.136

library(epiR)

# rows = algorithm (+/-), columns = truth (D+/D-).
tab <- as.table(matrix(c(261, 39, 18, 282), nrow = 2, byrow = TRUE,
                       dimnames = list(Algorithm = c("pos", "neg"),
                                       Truth     = c("Dpos", "Dneg"))))
res <- epi.tests(tab, conf.level = 0.95)
res$detail[res$detail$statistic %in% c("pv.pos", "pv.neg"), ]   # PPV and NPV with 95% CIs

# Project to a target prevalence using prevalence-invariant sensitivity & specificity.
project_pv <- function(sens, spec, prev) {
  ppv <- (sens * prev) / (sens * prev + (1 - spec) * (1 - prev))
  npv <- (spec * (1 - prev)) / (spec * (1 - prev) + (1 - sens) * prev)
  c(ppv = ppv, npv = npv)
}
project_pv(sens = 0.935, spec = 0.879, prev = 0.02)             # PPV ~ 0.136 in a rare cohort