EHR Phenotyping Algorithms

In plain language

An EHR phenotyping algorithm is a set of rules that scans a patient's electronic health record — their diagnosis codes, lab results, and prescriptions — and decides whether that patient counts as having a specific condition for a study. Instead of trusting a single billing code (which may have been entered just to justify a test), a phenotype combines multiple pieces of evidence so the definition is more accurate. Because no algorithm is perfect, researchers always check their rules against a small sample of real patient charts to measure how often the algorithm is right — a number called the positive predictive value.

An EHR phenotyping algorithm (computable phenotype) is the operational rule that turns raw clinical data into a binary (or graded) statement that a specific patient has — or does not have — a condition of interest, as of a specific date. In RWE it is the machinery behind cohort entry criteria, exposure proxies, covariates, and especially outcome ascertainment: the place where a study's internal validity is most often won or lost. A phenotype is not a code list; it is a code list plus logic (counts, positions, time windows, confirmatory labs/medications, exclusions) plus a measured operating characteristic (PPV, sensitivity, specificity) against a reference standard.

Core conceptual distinction

— two things are separable and must not be conflated. (1) The definition — the computable rule (e.g., "≥1 inpatient OR ≥2 outpatient ICD codes ≥30 days apart, plus a confirming HbA1c ≥6.5% or an antidiabetic dispensing"). (2) The performance — how that rule behaves against a validated reference (chart review, adjudication, registry, or a trusted linked source), expressed as positive predictive value (PPV), sensitivity, and specificity in the target population and era. The estimand the phenotype feeds also matters: a phenotype used to count incident outcome events needs high PPV and a defensible incident-date rule, whereas a phenotype used to define a denominator/cohort trades off sensitivity (completeness) against PPV (purity). The same code list can be an excellent outcome definition and a poor cohort definition. Crucially, error in a phenotype is measurement error / misclassification, and its direction matters: nondifferential outcome misclassification of a binary outcome biases a risk ratio toward the null, but differential misclassification by exposure (e.g., the exposed are surveilled more, so their outcomes are coded more completely) can bias in either direction and is not "conservative."

Pros, cons, and trade-offs

- vs a single-code "any diagnosis" definition: A multi-component phenotype (counts + positions + confirmatory lab/Rx + exclusions) dramatically raises PPV and is defensible to FDA/EMA/HTA reviewers. Cost: more programming, more judgment-dependent thresholds, and reduced sensitivity (you discard true cases that lack the confirmatory element). Prefer the richer rule for any consequential safety, effectiveness, or regulatory-grade outcome; reserve the single-code rule for hypothesis generation. - vs rule-based phenotyping with NLP / machine-learning phenotyping (e.g., PheNorm, APHRODITE, high-throughput models): ML/NLP phenotypes can recover cases that structured codes miss (notes mention a condition never coded) and scale across sites. Cost: they are harder to transport, harder to explain to a regulator, and can encode site-specific documentation artifacts; a model trained where the exposed are documented more will manufacture differential misclassification. Prefer transparent rule-based phenotypes for regulatory submissions and multi-site networks (PheKB-style portable logic); reserve NLP/ML for conditions poorly captured in structured data, and validate them per-site. - vs trusting an adjudicated registry/endpoint directly: Adjudicated outcomes are the reference standard and remove ascertainment ambiguity. Cost: registries are expensive, incomplete for exposure/follow-up, and may not cover the full source population. Prefer the registry as the validation gold standard and for adjudicated outcomes; use the EHR/claims phenotype for the scalable cohort and follow-up, ideally anchored to a validation substudy that estimates PPV/sensitivity for bias correction.

When to use

— any RWE study where the condition, exposure proxy, covariate, or outcome must be derived from routinely collected EHR/claims data; when no adjudicated source exists for the full cohort; when you need a portable, reproducible definition across sites or databases; and whenever a quantitative bias analysis will use measured PPV/sensitivity to correct for misclassification. A phenotype is mandatory when the endpoint is the study's primary outcome — pair it with a validation substudy and a pre-specified incident-event date rule.

When NOT to use — and when it is actively misleading or dangerous

- No validation and no plausible reference. Reporting an effect estimate from an unvalidated phenotype with unknown PPV is uninterpretable; if you cannot estimate operating characteristics in the study era and population, the algorithm is a black box, not a measurement. - Differential misclassification by exposure is plausible and unaddressed. If the exposed are seen, tested, or documented more (surveillance/detection bias — e.g., a drug requiring lab monitoring), a phenotype that depends on testing will over-ascertain outcomes in the exposed and fabricate an association. This is the dangerous case: the bias is not toward the null and balance tables will not reveal it. Use exposure-blind ascertainment, negative-control outcomes, and quantitative bias analysis. - Borrowing a phenotype across eras or data models without re-validation. ICD-9→ICD-10 transition, coding-incentive changes (e.g., risk-adjustment coding inflation), a new EHR vendor, or a new care-delivery model can move PPV by tens of points. A phenotype validated in one database/year is a hypothesis, not a fact, in another. - High-sensitivity cohort rule used as a high-PPV outcome rule (or vice versa). Using a loose "any code" denominator rule to count events inflates the numerator with false positives; using a strict confirmatory rule to define a denominator discards real patients and distorts incidence.

Data-source operational depth

- Claims (FFS vs Medicare Advantage): Only billed encounters generate codes, so phenotypes ride on reimbursement behavior, not clinical truth. Confirmatory labs and vitals are usually absent from claims (no result values), so you lean on the 1-inpatient / 2-outpatient (1-IP/2-OP) rules plus drug dispensings as proxies. Failure modes: MA-only person-time often lacks complete FFS-style encounter claims in older research extracts, so a "no diagnosis" patient may be unobserved rather than truly negative — restrict to enrollees with full medical+pharmacy benefit or treat MA spans as missing. Rule-out codes (a diagnosis billed to justify a test that came back negative) inflate false positives, which is why the 2-OP-codes-≥30-days-apart rule exists. Claims lag, reversals, and bundled/capitated services drop or delay codes; require continuous enrollment so absence of a code is informative. - EHR: Richer (labs with values, vitals, problem lists, orders, notes) so confirmatory logic (HbA1c ≥6.5%, eGFR, a positive culture) and NLP become possible — a genuine advantage over claims. But capture is encounter-driven and leaky: care delivered outside the system (an ER visit at another hospital, an outside lab) is invisible, so a patient who "leaves" looks event-free. Site workflow variation (who codes, which template, structured vs free-text) makes a single rule behave differently per site; problem lists are notoriously stale (conditions never resolved). Define observation windows explicitly and treat loss to follow-up as potentially informative. - Registry: Strongest for adjudicated, clinically rich case definitions (cancer stage, MI by universal definition) — the natural reference standard — but typically incomplete for full exposure history and longitudinal follow-up, and limited to the registered population. Use it to validate the EHR/claims phenotype and to supply adjudicated outcomes; link to claims for fills and to a death index for censoring. - Linked claims–EHR–registry/vital-records: The ideal substrate: EHR labs/notes sharpen the rule, claims fill capture gaps (outside care that was billed), and registry/death records adjudicate. Cost: linkage selects the linkable subset (selection bias), and order/result/service-date discrepancies must be reconciled before assigning an incident-event date. In the elderly, differential competing risks (death by exposure group) interact with phenotyping: a group that dies sooner has less opportunity to accrue the coded outcome, so naive cause-specific counts understate true burden — model competing risks explicitly when the phenotype feeds an incidence estimand.

Worked claims example (incident type 2 diabetes outcome phenotype + PPV/sensitivity validation). Goal: ascertain incident T2DM as an outcome during follow-up in a commercial+Medicare FFS database, then validate it. (1) Code logic: a case requires ≥1 inpatient T2DM diagnosis (ICD-10 E11.x in any position) OR ≥2 outpatient T2DM diagnoses on different dates ≥30 days apart (the 1-IP/2-OP rule; the 30-day separation suppresses single "rule-out" codes), OR ≥1 outpatient T2DM diagnosis plus a dispensing of a non-metformin antidiabetic within 60 days. (2) Incident-date rule: event date = the first qualifying code, and incidence requires a clean lookback — 365 days of continuous medical+pharmacy enrollment with no T2DM code and no antidiabetic dispensing before that date (washout makes the case incident, not prevalent; MA-only spans in the lookback are treated as unobserved, not as "no disease"). (3) Exposure-blind ascertainment: apply the identical rule and lookback to both treatment arms so any surveillance differences are not amplified. (4) Validation substudy: sample N=200 algorithm-positive charts; chart review (reference standard) confirms 170 true cases → PPV = 170/200 = 0.85. Separately, among a sample with adjudicated/linked truth you find the algorithm flagged 170 of 200 true cases → sensitivity = 0.85 (and specificity from the true-negatives). (5) Bias correction: with PPV and sensitivity, correct the observed counts (e.g., true positives = observed-positive × PPV; expand for missed cases using sensitivity) and propagate uncertainty via a quantitative bias analysis. (6) Sensitivity analyses: vary the OP-code window (30 vs 90 days), require vs drop the confirmatory Rx, re-estimate PPV in the ICD-10 era separately, and run a negative-control outcome to detect residual differential ascertainment.

Interpreting the output

. The output of an EHR phenotyping algorithm is an algorithm-positive cohort with documented operating characteristics. In the T2DM example, the algorithm returned 200 flagged cases; chart review confirmed 170 true cases, yielding PPV = 0.85. This means 85 out of every 100 patients flagged by this rule genuinely had incident T2DM in the study population and era — 15 were false positives driven by rule-out codes or documentation noise.

Formal interpretation: the algorithm produces a cohort of incident-T2DM candidates where PPV = 0.85 (95% CI from the 200-chart substudy). Sensitivity — the share of all true T2DM cases in the database that the algorithm captured — was separately estimated at 0.85 in a gold-standard-linked subsample. These two statistics are conceptually independent: a PPV of 0.85 from chart review of flagged cases tells you nothing directly about sensitivity; the two must be estimated from different samples or analytic designs. Nondifferential misclassification from a PPV below 1.0 biases risk-ratio estimates toward the null; differential misclassification (if the exposed are surveilled more) can bias in either direction.

Practical interpretation: before proceeding to any comparative analysis, apply bias correction — true positives ≈ observed positive × PPV — and quantify residual uncertainty via a Monte Carlo bias analysis. For regulatory or HTA submissions, document the PPV substudy design (sampling frame, reviewer credentials, adjudication rules) alongside the algorithm code in the SAP appendix.

Worked example

Scenario

A researcher wants to identify patients newly diagnosed with type 2 diabetes in a hospital's EHR. Rather than accepting any single billing code at face value, the team writes a rule-based phenotype that requires three things to be true at the same time: (1) at least one ICD-10 code for type 2 diabetes on record, (2) a hemoglobin A1c lab result at or above 6.5 percent, and (3) a prescription for a diabetes medication. Five patients are in the candidate pool. The table below shows which criteria each patient meets and whether the algorithm flags them.

Dataset

Candidate patients and which phenotype criteria each meets (Y = yes, N = no).

Steps

• The algorithm requires ALL THREE criteria: a type 2 diabetes ICD-10 code AND an HbA1c lab result at or above 6.5 percent AND an active diabetes prescription. This is rule-based phenotyping — the logic is written out as explicit if-then conditions with no statistical model involved.

• P001 has all three: a diabetes code, a qualifying HbA1c, and a diabetes drug — flagged.

• P002 has a diabetes code and a prescription but the HbA1c is below 6.5 percent, so the lab criterion fails — not flagged.

• P003 has a qualifying HbA1c and a prescription but no diabetes code at all — not flagged.

• P004 has a diabetes code and a qualifying HbA1c but no prescription on record — not flagged.

• P005 meets all three criteria — flagged.

• Result: 2 patients (P001, P005) are flagged out of 5 candidates.

• A probabilistic approach would instead train a model to score each patient from 0 to 1 and pick a cutoff — useful when conditions rarely appear in structured codes and notes must be read — but harder to explain to a regulator than the transparent rule above.

• To validate the rule, the team randomly selects the 2 flagged patients (and perhaps some non-flagged ones) and has a clinician read their actual charts. If both flagged patients are confirmed as true type 2 diabetics, the PPV = 2 true positives / 2 flagged = 1.00, meaning the algorithm was correct every time it fired. With more patients, a realistic PPV might be 0.85, meaning 15 percent of flags are false alarms — which the research team would report and could correct for mathematically.

Result

2 of 5 candidates are flagged (P001 and P005). Both meet all three criteria. If a clinician confirms both are true type 2 diabetics, PPV = 2/2 = 1.00 in this small sample. In a realistic study with hundreds of flagged patients, PPV is estimated from a chart-review sample of around 200 and typically falls between 0.80 and 0.90 for a well-designed multi-criterion rule.

Runnable example

import pandas as pd
import numpy as np

T2DM = ("E11",)                 # ICD-10 prefix for type 2 diabetes
WASHOUT_DAYS = 365              # disease-free + continuous-enrollment lookback -> incident, not prevalent
OP_GAP_DAYS = 30               # two outpatient codes must be >= this far apart (suppress rule-out codes)
RX_CONFIRM_DAYS = 60          # confirmatory dispensing window after a single OP code

def is_t2dm(code: str) -> bool:
    return code.startswith(T2DM)

def build_t2dm_phenotype(dx: pd.DataFrame, rx: pd.DataFrame,
                         enroll: pd.DataFrame) -> pd.DataFrame:
    dx = dx[dx["icd_code"].map(is_t2dm)].sort_values(["person_id", "dx_date"])

    # Rule A: >=1 inpatient T2DM code.
    ip = dx[dx["setting"] == "IP"].groupby("person_id")["dx_date"].min()

    # Rule B: >=2 outpatient T2DM codes on dates >= OP_GAP_DAYS apart.
    op = dx[dx["setting"] == "OP"]
    def second_op_date(g):
        d = g["dx_date"].sort_values().to_numpy()
        for i in range(1, len(d)):
            if (d[i] - d[0]) >= np.timedelta64(OP_GAP_DAYS, "D"):
                return pd.Timestamp(d[i])
        return pd.NaT
    op_qual = op.groupby("person_id").apply(second_op_date).dropna()

    # Rule C: >=1 OP code + a confirming non-metformin antidiabetic dispensing within RX_CONFIRM_DAYS.
    first_op = op.groupby("person_id")["dx_date"].min()
    rxc = rx[rx["drug_class"] == "ANTIDIABETIC_NONMET"]
    cm = first_op.reset_index().merge(rxc, on="person_id", how="inner")
    cm = cm[(cm["fill_date"] >= cm["dx_date"]) &
            (cm["fill_date"] <= cm["dx_date"] + pd.Timedelta(days=RX_CONFIRM_DAYS))]
    rx_qual = cm.groupby("person_id")["dx_date"].min()

    # Earliest qualifying date across rules = candidate incident event date.
    cand = pd.concat([ip.rename("d"), op_qual.rename("d"), rx_qual.rename("d")])
    event = cand.groupby(level=0).min().rename("event_date").reset_index()
    event.columns = ["person_id", "event_date"]

    # Incidence: clean 365d lookback (no prior T2DM code, no antidiabetic), continuous non-MA enrollment.
    prior_dx = dx.merge(event, on="person_id")
    had_prior = prior_dx[(prior_dx["dx_date"] < prior_dx["event_date"]) &
                         (prior_dx["dx_date"] >= prior_dx["event_date"] -
                          pd.Timedelta(days=WASHOUT_DAYS))]["person_id"].unique()
    e = enroll.merge(event, on="person_id")
    covered = e[(e["enroll_start"] <= e["event_date"] - pd.Timedelta(days=WASHOUT_DAYS)) &
                (e["enroll_end"] >= e["event_date"]) & (~e["ma_only"])]["person_id"].unique()

    positive = event[event["person_id"].isin(covered) &
                     ~event["person_id"].isin(had_prior)].copy()
    positive["algo_positive"] = True
    return positive[["person_id", "event_date", "algo_positive"]]

def validate(positive: pd.DataFrame, gold: pd.DataFrame) -> dict:
    # PPV: among algorithm-positive charts reviewed, fraction that are true cases.
    flagged = gold.merge(positive[["person_id"]], on="person_id", how="left",
                         indicator=True)
    flagged["algo_positive"] = flagged["_merge"] == "both"
    tp = int(((flagged["algo_positive"]) & (flagged["true_case"])).sum())
    fp = int(((flagged["algo_positive"]) & (~flagged["true_case"])).sum())
    fn = int(((~flagged["algo_positive"]) & (flagged["true_case"])).sum())
    tn = int(((~flagged["algo_positive"]) & (~flagged["true_case"])).sum())
    return {
        "ppv": tp / (tp + fp) if (tp + fp) else np.nan,
        "sensitivity": tp / (tp + fn) if (tp + fn) else np.nan,
        "specificity": tn / (tn + fp) if (tn + fp) else np.nan,
        "n_reviewed": tp + fp + fn + tn,
    }

library(data.table)
WASHOUT_DAYS    <- 365L
OP_GAP_DAYS     <- 30L
RX_CONFIRM_DAYS <- 60L

build_t2dm_phenotype <- function(dx, rx, enroll) {
  setDT(dx); setDT(rx); setDT(enroll)
  dx <- dx[grepl("^E11", icd_code)][order(person_id, dx_date)]

  # Rule A: >=1 inpatient code.
  ip <- dx[setting == "IP", .(d = min(dx_date)), by = person_id]

  # Rule B: >=2 outpatient codes >= OP_GAP_DAYS apart (earliest qualifying second date).
  op <- dx[setting == "OP"]
  op_qual <- op[, {
    d <- sort(dx_date)
    hit <- if (length(d) > 1L) d[which(d - d[1L] >= OP_GAP_DAYS)] else as.Date(character())
    .(d = if (length(hit)) min(hit) else as.Date(NA))
  }, by = person_id][!is.na(d)]

  # Rule C: >=1 OP code + non-metformin antidiabetic fill within RX_CONFIRM_DAYS.
  first_op <- op[, .(dx_date = min(dx_date)), by = person_id]
  rxc <- rx[drug_class == "ANTIDIABETIC_NONMET"]
  cm  <- merge(first_op, rxc, by = "person_id")
  rx_qual <- cm[fill_date >= dx_date & fill_date <= dx_date + RX_CONFIRM_DAYS,
                .(d = min(dx_date)), by = person_id]

  cand  <- rbindlist(list(ip, op_qual, rx_qual))
  event <- cand[, .(event_date = min(d)), by = person_id]

  # Incidence: clean 365d lookback + continuous non-MA enrollment through event date.
  pd <- merge(dx, event, by = "person_id")
  had_prior <- unique(pd[dx_date < event_date &
                         dx_date >= event_date - WASHOUT_DAYS, person_id])
  e <- merge(enroll, event, by = "person_id")
  covered <- unique(e[enroll_start <= event_date - WASHOUT_DAYS &
                      enroll_end   >= event_date & !ma_only, person_id])

  pos <- event[person_id %chin% covered & !person_id %chin% had_prior]
  pos[, algo_positive := TRUE]
  pos[, .(person_id, event_date, algo_positive)]
}

validate_phenotype <- function(positive, gold) {
  setDT(gold)
  gold[, algo_positive := person_id %chin% positive$person_id]
  tp <- gold[algo_positive  & true_case,  .N]
  fp <- gold[algo_positive  & !true_case, .N]
  fn <- gold[!algo_positive & true_case,  .N]
  tn <- gold[!algo_positive & !true_case, .N]
  list(ppv = tp / (tp + fp), sensitivity = tp / (tp + fn),
       specificity = tn / (tn + fp), n_reviewed = tp + fp + fn + tn)
}