EHR-Based Study

In plain language

An EHR-based study answers a clinical question using the electronic health record a hospital or clinic fills in while caring for patients — the doctor's orders, the clinic visits, the lab and vital-sign results, and the typed notes. Its big advantage over insurance billing data is that it can see actual measurements claims never carry, like a kidney-function lab or a blood pressure. Its big catch is that the record only exists when the patient shows up: a written drug order is just intent, not proof the patient ever filled or took it, and any care that happens outside this health system is simply invisible.

An EHR-based study is a non-interventional study whose entire data substrate is the electronic health record — provider orders and medication administrations, encounter and visit records, problem lists, structured laboratory and vital-sign values, and free-text clinical notes — captured as a by-product of clinical care. It is a study design type in the sense of "what generated the data and therefore what biases are baked in," not an estimation method: the design choice dictates how time zero is defined, how the cohort is observed, and which threats to validity dominate. The analytic methods layered on top (new-user/active-comparator restriction, propensity scores, Cox models) are the same as in any cohort study; what is distinctive is that the EHR data-generating process — driven by clinical encounters, not insurance enrollment — reshapes nearly every operational decision.

Core conceptual distinction

The defining contrast is EHR vs administrative claims as the source of person-time and ascertainment. Claims are a transaction stream tied to insurance enrollment: a clean enrollment table tells you exactly when a person is observable, pharmacy claims capture dispensings with `days_supply`, and capture is near-complete within the enrolled window but blind to anything outside it (cash purchases, care under a different plan, clinical detail). EHR is an encounter stream tied to where a patient seeks care: it carries rich clinical granularity (labs, vitals, problem lists, notes, severity) that claims never see, but it has no enrollment table — a patient is "observable" only when they show up, and silence is ambiguous (healthy? died? went elsewhere?). Three primitives shift accordingly. (1) Exposure in EHR is the medication order or administration, which is an intent or in-hospital event, not proof the patient filled and took the drug; confirming initiation usually requires linkage to pharmacy fills. (2) Observability must be constructed from encounter activity (e.g., at least one visit in a trailing window) rather than read off enrollment spans. (3) Phenotyping — turning raw EHR signals into an exposure/outcome/covariate — is a research artifact in itself, combining structured codes (ICD, SNOMED, RxNorm, LOINC) with problem-list curation and, increasingly, NLP over notes, and must be validated against a reference standard (chart review or PPV estimates). The estimand is unchanged from any cohort study (e.g., a comparative hazard ratio or risk difference), but its transportability is conditioned on the catchment of the health system that produced the records.

Pros, cons, and trade-offs

(named against the real alternatives). - vs claims-based studies: EHR wins decisively on clinical depth — measured labs (eGFR, HbA1c, LDL), vitals (BP, BMI), disease severity, smoking/functional status, and outcome detail (tumor stage, ejection fraction) that claims can only proxy. This directly improves confounding control and enables outcome phenotypes claims cannot support. Cons: EHR has no complete picture of utilization — care delivered outside the network is invisible, there is no enrollment denominator, and pharmacy capture is order-level (intent), not dispensing-level. Prefer EHR when the question hinges on clinical measurements or fine-grained outcomes; prefer claims when complete drug exposure, full utilization, and a defined denominator are paramount; prefer linked EHR-claims when you can get both. - vs registries: Registries give adjudicated, protocol-defined outcomes and curated severity for a specific disease, but are narrow and expensive. EHR is broader and cheaper but its variables are care-driven and non-adjudicated (informed presence: sicker patients generate more data). Prefer registries for adjudicated endpoints; prefer EHR for breadth and rapid feasibility. - vs linked claims-EHR-vital-records: Linkage is the ideal substrate (EHR severity + claims completeness + reliable death) but it costs a selection layer (only the linkable subset) and date-reconciliation between order, fill, and service dates. Prefer linkage when mortality or complete exposure is decisive and a linkable cohort is available.

When to use

The exposure or outcome requires clinical detail that claims cannot supply (lab-based kidney/glycemic/lipid endpoints, BP/BMI/severity, imaging or pathology findings, in-hospital medication administration); you need outcome phenotypes built from labs/notes; you are doing rapid feasibility, hypothesis generation, or signal detection within a defined health system; or you have linkage that lets EHR severity ride on top of claims completeness.

When NOT to use — and when it is actively misleading or dangerous

- The question needs complete drug exposure or a denominator. Drug-utilization, adherence (PDC/MPR), or incidence-rate questions are dangerous in stand-alone EHR: orders are not fills, out-of-network fills are invisible, and there is no enrollment denominator — you will misclassify exposure and cannot compute person-time correctly. Use claims (or link to them). - Informed-presence / surveillance bias is differential by exposure. If one drug's patients are monitored more intensely, their outcomes are detected more often purely because they are seen more — a spurious association. EHR makes this worse than claims because ascertainment is visit-driven. - Loss to follow-up is informative and differential. A patient who improves, dies, or transfers care simply stops generating records; treating that silence as continued event-free follow-up induces immortal-time-like and survivorship distortions. If you cannot anchor censoring (e.g., a linked death index), event rates are uninterpretable. - A single-system EHR is being used to answer a population question. The catchment defines the population; effect estimates may not transport beyond that health system's case-mix and practice patterns. - Unvalidated phenotypes. Deploying an ICD- or NLP-based outcome definition without PPV/sensitivity estimates against a reference standard bakes in differential misclassification of unknown magnitude.

Data-source operational depth

- Claims (FFS or MA/commercial): Used here as the comparator substrate and as the linkage target. Strength: complete dispensing (`days_supply`), full cross-provider utilization within enrollment, a real denominator. Failure modes: MA encounter data are often incomplete relative to Medicare FFS (capitated providers underreport), so MA-only person-time can masquerade as low utilization; cash and sample drugs are invisible; coding intensity varies by payer. Workaround when linked: restrict to FFS (Parts A/B/D) for exposure/utilization completeness and use the EHR only for clinical detail. - EHR: Strength: labs, vitals, problem lists, notes, severity, in-hospital administrations. Failure modes: (a) No enrollment table — construct observability from encounter activity (e.g., ≥1 encounter in the prior 365 days) and accept that "new user" really means "first observed order"; (b) orders ≠ dispensings — prefer linked fills to confirm initiation; (c) informed-presence / differential surveillance — adjust for visit frequency or use negative controls; (d) fragmented records across systems — patients seen elsewhere look like gaps; (e) competing risks and death — out-of-hospital deaths are frequently uncaptured, so link to a death index before trusting censoring. In elderly/oncology cohorts, differential competing risks by exposure (e.g., one arm with sicker patients dying of other causes) demand a Fine-Gray or cause-specific framing, not naive Kaplan-Meier on the EHR alone. - Registry: Adjudicated outcomes and curated severity for a defined disease; weak for complete pharmacy exposure and general comorbidity. Link to claims for fills and to a death index for censoring. - Linked claims-EHR-vital-records: Ideal but introduces linkage selection (only the linkable subset, which differs systematically) and order/fill/service date discrepancies that must be reconciled before time-zero assignment; an order on 2024-01-03 with a fill on 2024-01-06 cannot both be time zero.

Worked EHR example

Question: incident acute kidney injury (AKI) after initiating an SGLT2 inhibitor in adults with type 2 diabetes, using a multi-hospital EHR linked to pharmacy fills. (1) Observability (the claims-style enrollment substitute): require ≥1 outpatient encounter in the 365 days before the index order, so the patient is genuinely "in" the system and a prior order would have been seen. (2) Exposure / time zero: the first order of an SGLT2 inhibitor with no prior order of any SGLT2 inhibitor in the 365-day lookback (new-user restriction on observed orders); require a linked pharmacy `fill_date` within 30 days of the order to confirm the patient actually started — index_date = the order date, but only for confirmed starters. (3) Baseline phenotype & covariates: measured only in `[index_date - 365, index_date]` — most recent eGFR and HbA1c from the structured lab table (a clinical variable claims cannot supply), BP/BMI from vitals, diabetes complications and comorbidities from ICD diagnoses + problem list, and an explicit visit-count covariate to blunt informed-presence bias. (4) Outcome phenotype: AKI defined by a validated rule — a ≥1.5× rise in serum creatinine within 7 days (KDIGO lab criterion) OR an AKI diagnosis code at an encounter — with the PPV of the rule estimated against chart review before it is trusted. (5) Follow-up & censoring: from time zero to first AKI, censoring at the last observed encounter + a grace window (the EHR equivalent of disenrollment), at linked death (out-of-hospital deaths come from the death index, not the EHR), and at end of data; for an as-treated analysis, censor at the end of the last confirmed fill's `days_supply` + grace. (6) Analysis & diagnostics: PS-adjust on the baseline window, fit a cause-specific or Fine-Gray model treating death as a competing risk, and run sensitivity analyses on the observability window length, the order-to-fill confirmation requirement, and a negative-control outcome to surface residual surveillance bias.

Worked example

Scenario

Meet patient 4021, a 58-year-old with type 2 diabetes seen at one hospital system. Below is a slice of her EHR for a single day in March. We want to read these mixed rows the way an analyst would and see exactly what the EHR shows — and what it quietly hides compared with insurance claims.

Dataset

A handful of raw EHR rows for one patient on one day, mixing the different record types an analyst actually pulls from separate EHR tables into one view.

Steps

• Read the rows by type: one encounter anchors the day, two labs and one vital give measured clinical detail, one order records a prescribing decision, and one note is typed free text.

• Notice what claims could never give you here: an eGFR of 62, an HbA1c of 8.4%, and a blood pressure of 138/86 are real measurements — insurance billing data would at best show a diagnosis code, never the number.

• Now read the order carefully: 'empagliflozin ordered' means the clinician intended to start the drug, but there is no pharmacy fill row on this day — an order is not a fill, so we cannot yet say she actually began taking it.

• To confirm she truly started, you would need a linked pharmacy fill within a short window after 2024-03-12; without it, treating this order as a started treatment would overcount real initiation.

• Finally, picture the weeks after this visit with no new rows. In claims that gap could mean she stopped care; in the EHR it could also just mean she filled the drug at an outside pharmacy and saw a doctor in another system — care this hospital's EHR cannot see, so she can look lost to follow-up while doing perfectly fine.

Result

For patient 4021 on 2024-03-12 the EHR shows a real endocrinology visit with measured kidney function (eGFR 62), glycemic control (HbA1c 8.4%), and blood pressure (138/86), plus a clinician's order to start an SGLT2 inhibitor and a supporting note — clinical depth that insurance claims simply do not carry. The key limitation: that order is only intent, not a confirmed fill, and because the EHR records data only when this system sees the patient, any later fills or visits made elsewhere are invisible, so a silent stretch in the record can be mistaken for the patient disappearing or going untreated.

Runnable example

import pandas as pd

LOOKBACK_DAYS = 365   # observability + drug-free lookback (EHR substitute for continuous enrollment)
FILL_GRACE    = 30    # days after the order within which a linked fill confirms true initiation

def build_ehr_new_user_cohort(orders, encounters, fills):
    orders = orders.sort_values(["person_id", "order_date"])

    # Candidate index = first STUDY-drug order per person.
    study = orders[orders["drug_class"] == "STUDY"]
    idx = (study.groupby("person_id", as_index=False)
                .first()
                .rename(columns={"order_date": "index_date"})
                [["person_id", "index_date"]])

    # Observability: require >=1 encounter in the LOOKBACK window before index (no enrollment table exists).
    enc = encounters.merge(idx, on="person_id")
    observable = enc[(enc["encounter_date"] < enc["index_date"]) &
                     (enc["encounter_date"] >= enc["index_date"] - pd.Timedelta(days=LOOKBACK_DAYS))]
    idx = idx[idx["person_id"].isin(observable["person_id"])].copy()

    # New-user: no STUDY order in the LOOKBACK window before index (first *observed* order).
    prior = study.merge(idx, on="person_id")
    prior_in_lb = prior[(prior["order_date"] < prior["index_date"]) &
                        (prior["order_date"] >= prior["index_date"] - pd.Timedelta(days=LOOKBACK_DAYS))]
    idx = idx[~idx["person_id"].isin(prior_in_lb["person_id"])].copy()

    # Confirm initiation: a linked fill of the same class within FILL_GRACE days of the order.
    f = fills[fills["drug_class"] == "STUDY"].merge(idx, on="person_id")
    confirmed = f[(f["fill_date"] >= f["index_date"]) &
                  (f["fill_date"] <= f["index_date"] + pd.Timedelta(days=FILL_GRACE))]
    cohort = idx[idx["person_id"].isin(confirmed["person_id"])].copy()

    cohort["baseline_start"] = cohort["index_date"] - pd.Timedelta(days=LOOKBACK_DAYS)
    return cohort[["person_id", "index_date", "baseline_start"]]

library(data.table)
LOOKBACK_DAYS <- 365L
FILL_GRACE    <- 30L

build_ehr_new_user_cohort <- function(orders, encounters, fills) {
  setDT(orders); setDT(encounters); setDT(fills)
  setorder(orders, person_id, order_date)

  study <- orders[drug_class == "STUDY"]
  idx <- study[, .(index_date = order_date[1L]), by = person_id]

  # Observability: >=1 encounter in the lookback window before index (EHR has no enrollment span).
  enc <- merge(encounters, idx, by = "person_id")
  obs_ids <- unique(enc[encounter_date < index_date &
                        encounter_date >= index_date - LOOKBACK_DAYS, person_id])
  idx <- idx[person_id %chin% obs_ids]

  # New-user: no STUDY order in the lookback window before index (first observed order).
  study <- merge(study, idx, by = "person_id")
  prior_ids <- unique(study[order_date < index_date &
                            order_date >= index_date - LOOKBACK_DAYS, person_id])
  idx <- idx[!person_id %chin% prior_ids]

  # Confirm initiation via a linked fill within the grace window after the order.
  f <- merge(fills[drug_class == "STUDY"], idx, by = "person_id")
  conf_ids <- unique(f[fill_date >= index_date &
                       fill_date <= index_date + FILL_GRACE, person_id])

  cohort <- idx[person_id %chin% conf_ids]
  cohort[, baseline_start := index_date - LOOKBACK_DAYS]
  cohort[, .(person_id, index_date, baseline_start)]
}