Patient-Reported Outcomes in Real-World Settings

In plain language

A patient-reported outcome (PRO) is a score that comes directly from the patient — how much pain they feel, how well they can walk, how tired they are — rather than from a lab test or a doctor's chart note. In real-world research, these scores are collected outside a clinical trial: through a patient portal, a disease registry, or a tablet in a clinic waiting room, so the data is messier and more often missing than in a trial. The goal is to learn whether a treatment makes patients feel or function better by a meaningful margin, which researchers call the minimally important difference (MID) — the smallest change in score that a patient would actually notice.

A real-world PRO is a self-reported measure of a patient's health status — symptoms, function, or health-related quality of life — captured outside a protocol-driven trial: in a disease registry, an ePRO app or patient portal, or embedded in EHR workflows (e.g., PROMIS computer-adaptive tests pushed before oncology visits). The instrument may be identical to the one used in a trial; what changes is the data-generating process. In a trial, assessment timing is scheduled, completion is chased by coordinators, and the analysis population is fixed at randomization. In the real world, assessment is visit-driven or opt-in, completion is voluntary and self-selected, and the patients who keep responding differ systematically from those who stop — usually because the sickest patients are the ones who drop out. This entry is about the measurement-and-analysis engineering required to turn that messy stream of submissions into a defensible analytic dataset; it is distinct from instrument creation (`pro-development`) and psychometric validation (`pro-validation`), which assume the instrument already performs and ask a different question.

Core conceptual distinction

. Three things separate real-world PRO measurement from trial PRO measurement, and each drives a specific analytic decision. (1) Assessment timing is endogenous. A real-world PRO is recorded when the patient shows up or chooses to respond, not on a fixed schedule, so you must impose an explicit completion window (e.g., a scheduled landmark ±14 days) and a time-zero anchor (initiation, diagnosis, or surgery date — often taken from claims, not from the PRO system) to make scores from different patients comparable. (2) Missingness is informative. Non-response in routine care is almost never missing-completely-at-random: it correlates with the outcome itself (a patient too sick or too symptomatic to complete a questionnaire), making naive complete-case or last-observation-carried-forward (LOCF) analyses biased toward a falsely stable or falsely improving trajectory. (3) The cohort is a completer cohort. The patients with usable PRO data are a self-selected subset of the enrolled or treated population, so the estimand silently shifts from "the PRO trajectory of treated patients" to "the PRO trajectory of patients who keep filling out questionnaires." Stating which of these you mean — and whether you are estimating a within-person change, a between-group comparison, or a responder proportion — is the estimand discipline this concept demands.

Pros, cons, and trade-offs

. - vs trial-collected PROs: Real-world PROs deliver longer follow-up, larger and more representative populations, and capture of subgroups trials exclude (frail, multimorbid, very old). Cost: timing is irregular, completion is low and selective, and there is no coordinator chasing missing forms — so the missing-data and selection problems that are nuisances in a trial become the central methodological threat. Prefer real-world PROs when the question is durability, real-world burden, or HTA value evidence; prefer trial PROs when you need an unbiased treatment-effect estimate on a PRO endpoint. - vs clinician-reported or claims-derived outcomes: PROs capture the patient's lived experience (fatigue, pain, role function) that claims and chart abstraction systematically miss. Cost: they are subjective, vulnerable to response shift and recall, and far more missing than a claims-based event (an inpatient claim exists whether or not the patient feels like reporting). Prefer PROs for symptom/QoL endpoints; pair them with claims-confirmed clinical events (`endpoint-adjudication-chart-review-rwe`) so that the absence of a PRO is not mistaken for the absence of an event. - vs complete-case analysis of real-world PROs: Mixed-effects/MMRM models (`mmrm-repeated-measures-rwe`) under a missing-at-random assumption, with multiple imputation (`multiple-imputation-longitudinal-rwe`) and explicit MNAR sensitivity analysis, recover far less biased trajectories than complete-case or LOCF. Cost: more assumptions and code, and MAR is still an assumption that informative dropout can violate. Prefer the model-based approach; reserve complete-case only as a transparency check, never as the primary analysis.

When to use

. Estimating PRO trajectories, symptom burden, or HRQoL over routine follow-up; generating utility inputs for cost-utility models via QALY mapping (`qaly-utility-mapping-rwe`); HTA submissions where regulators expect real-world durability of patient-relevant benefit; pragmatic symptom-monitoring programs; characterizing the patient experience in registries where a trial is infeasible. The prerequisites are a validated instrument with an established minimally important difference (MID), a documentable completion window and time-zero anchor, and a missingness plan written before the analysis.

When NOT to use — and when it is actively misleading or dangerous

. - As an unadjusted treatment-effect estimate. Comparing PRO change between two real-world treatment groups without confounding control and differential-missingness handling is doubly biased: by indication and by who keeps responding. A drug that makes patients feel worse can look better simply because the patients it harms stop completing surveys. - With LOCF or complete-case as the primary analysis when dropout is informative. LOCF freezes the last (often pre-decline) score and manufactures stability; complete-case conditions on survival-to-response. Both are dangerous in progressive disease (oncology, ALS, advanced heart failure) where the patients who disappear are the ones declining. - When the completer cohort no longer maps to a meaningful population. If only 20% of an opt-in ePRO cohort responds at month 12 and responders are younger, English-speaking, and higher-SES, the surviving estimand describes a population no decision-maker cares about — report the completion funnel, do not bury it. - Across instruments or versions without a crosswalk. Pooling SF-36 with PROMIS, or PRO-CTCAE free-text with graded items, without a validated linkage produces a trajectory artifact of the instrument switch, not the patient. - When the "real-world" PRO is actually a single cross-sectional snapshot marketed as longitudinal evidence — a one-time portal survey cannot support a trajectory or responder claim.

Data-source operational depth

. - Disease/product registry (scheduled PRO at fixed visits): The best structure for comparability, but completion falls off with disease severity and at later visits, so missingness is informative; LOCF here hides genuine decline. Failure mode: registries often record only the score and visit, not why a form is missing (death? too sick? lost to follow-up?), so you cannot distinguish MAR from MNAR without linking to a mortality source (`mortality-source-hierarchy-rwe`) and claims for hospitalizations. Workaround: build an explicit assessment-expected vs assessment-observed table per visit and model dropout jointly or via MNAR sensitivity (delta-adjusted multiple imputation). - ePRO app / patient portal (opt-in, push-notification driven): Subject to a digital-divide selection — older, lower-SES, non-English, and less digitally literate patients under-respond — so the responder cohort is not the enrolled cohort. Push-notification timing and reminder cadence change who responds and when, contaminating the completion window. Failure mode: multiple submissions inside one window (a patient answers twice) and unscheduled "I feel awful today" submissions that oversample bad days. Workaround: pre-specify a window-collapse rule (e.g., the submission closest to the scheduled landmark; or first-in-window), cap one record per person-window, and report completion by demographic strata to expose the selection. - EHR-embedded PROMIS via portal (visit-driven): Captured only when the patient has a visit and the workflow fires, so there is no PRO when the patient leaves the system or is hospitalized elsewhere — visit-driven capture means departure is differential. Failure mode: a "stable" PRO trajectory can mask a claims-confirmed hospitalization that occurred between two portal completions. Workaround: link to claims/EHR encounter data so that clinical events between PRO assessments are observed, and treat a PRO gap that brackets a major event as informative, not ignorable. - Linked claims + PRO: The strongest substrate — claims supply the time-zero anchor (initiation/surgery date), confirm clinical events and death (so absence of a PRO is not read as absence of an event), and let you characterize non-responders on observable claims covariates to test the MAR assumption. Cost: linkage selection (only the linkable subset has both), and date reconciliation between the PRO timestamp and the claim service date before window assignment.

Worked example (linked registry ePRO + claims)

Question: the 12-month FACT-G total-score trajectory and 6-month responder rate after initiating a new oral oncolytic, among adults in a cancer registry with linked commercial + Medicare FFS claims. (1) Time zero: the first dispensing of the oncolytic from the pharmacy claim (`fill_date` on the index NDC), not the registry enrollment date, so all patients are aligned at treatment initiation. (2) Eligibility / observability: require 365 days of continuous medical + pharmacy enrollment before `index_date` (exclude Medicare Advantage-only person-time, which lacks FFS claims and would make the baseline window unobservable) and at least one PRO submission in the baseline window. (3) Baseline PRO: the FACT-G submission in the window [`index_date` − 30, `index_date` + 7]; if multiple submissions fall in-window, keep the one closest to `index_date` (window-collapse rule) and cap one record per person-window. (4) Scheduled landmarks: months 3, 6, 9, 12, each with a completion window of the scheduled day ±14 days; the same collapse and cap rules apply. (5) Responder definition: a patient is a 6-month responder if their month-6 FACT-G is ≥ their baseline plus the MID of 5 points (anchor-based MID; report deterioration symmetrically as a drop ≥5). (6) Informative missingness: before scoring, build an expected-vs-observed completion table by landmark and by age/SES/language stratum; classify each missing assessment using linked claims and a mortality source as death, hospitalization-near-window, disenrollment, or unexplained — because a patient who is hospitalized (claims-confirmed) at month 6 but has no PRO is informatively, not ignorably, missing. (7) Primary analysis: an MMRM on change-from-baseline with an unstructured covariance over the four landmarks (MAR), adjusting for baseline score and confounders measured in the 365-day claims lookback; the responder rate is reported under multiple imputation, never LOCF. (8) Sensitivity: delta-adjusted (MNAR) multiple imputation that penalizes imputed scores for dropouts, and a tabulated completion funnel so reviewers see exactly how the analytic cohort narrowed from enrolled to baseline-observed to month-12-observed.

Interpreting the output

Consider the worked example: mean PROMIS Physical Function change from baseline is 1.5 points at 6 months (sum of changes 5 + 2 − 6 + 5 = 6, divided by 4 patients), with 2 of 4 patients (50%) classified as responders (change ≥ 3 points, the MID) and 1 of 4 (25%) classified as deteriorators.

Formal interpretation: A mean change of 1.5 points is below the anchor-based MID of 3 points, meaning that on average the group did not achieve a patient-meaningful improvement — even though the arithmetic mean is positive. The mean and the responder rate tell complementary stories that neither tells alone: the mean of 1.5 masks that two patients improved by 5 points each while one deteriorated by 6. In routine-care PRO data, missing-not-at-random missingness is the dominant analytic threat: patients who are more symptomatic at month 6 are less likely to complete the questionnaire. If the 1.5-point mean was computed only among completers, it is likely biased upward — the deteriorating patient (P003, change −6) might not have submitted if the context were a real-world registry rather than a controlled example. Reporting completers-only estimates without a missingness model is therefore not interpretable as a population-level treatment effect.

Practical interpretation: Pair the mean change with the completion funnel (number enrolled, number with baseline PRO, number with 6-month PRO) and the MNAR sensitivity analysis results. If the delta-adjusted imputation moves the mean from 1.5 toward a smaller or negative number, the missingness is informative enough to change the clinical conclusion. Report both the responder rate and the deterioration rate: a 50% responder rate with a 25% deterioration rate conveys that the treatment's average effect hides substantial individual-level heterogeneity — a finding relevant to patient communication and subgroup analyses.

Worked example

Scenario

Four patients with a chronic inflammatory joint disease start a new oral treatment. Before treatment (baseline) and six months later, each patient completes the PROMIS Physical Function questionnaire, which scores physical ability from 0 to 100 (higher = better). The MID for this scale is 3 points: a change of at least +3 counts as meaningful improvement, and a change of -3 or worse counts as meaningful deterioration. We want to know the mean change in the group and how many patients were responders.

Dataset

PROMIS Physical Function T-scores for four patients at baseline and 6 months. A researcher would see one row per patient per time point in a registry or EHR export.

Steps

• Read the PRO scores: each row is one patient. The baseline_score is what the patient reported before starting treatment; the month6_score is what they reported six months later.

• Compute change_from_baseline for each patient by subtracting baseline from month6: P001 = 47 - 42 = +5, P002 = 40 - 38 = +2, P003 = 45 - 51 = -6, P004 = 41 - 36 = +5.

• Apply the MID rule: a change >= +3 means the patient is a responder (meaningful improvement); a change <= -3 means meaningful deterioration. P001 (+5) and P004 (+5) are responders. P002 (+2) improved but not enough to cross the MID — that is called stable. P003 (-6) deteriorated.

• Compute the mean change for the group: sum the four changes and divide by 4. Sum = 5 + 2 + (-6) + 5 = 6. Mean = 6 / 4 = 1.5 points.

• Check the mean against the MID: 1.5 is below the MID of 3, so on average the group did not improve by a clinically meaningful amount, even though the arithmetic average is positive.

• Summarize the responder analysis: 2 out of 4 patients (50%) were responders; 1 out of 4 (25%) deteriorated; 1 out of 4 (25%) was stable. Reporting these categories alongside the mean is important because the mean can hide that some patients got much better while others got worse.

Result

Mean change from baseline = 6 / 4 = 1.5 points. This is below the MID of 3 points, so the average group change is not clinically meaningful. Responder rate = 2 / 4 = 50% (patients P001 and P004). Deterioration rate = 1 / 4 = 25% (patient P003).

Runnable example

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf

LANDMARKS = {3: 90, 6: 182, 9: 274, 12: 365}   # month -> days from index
WINDOW = pd.Timedelta(days=14)                  # +/- completion window around each landmark
BASE_LO, BASE_HI = pd.Timedelta(days=30), pd.Timedelta(days=7)   # baseline window [-30, +7]
MID = 5.0                                        # FACT-G anchor-based minimally important difference

def build_pro_panel(pro: pd.DataFrame, index: pd.DataFrame, enroll: pd.DataFrame) -> pd.DataFrame:
    df = pro.merge(index[["person_id", "index_date"]], on="person_id")

    # Observability: continuous, FFS-observable enrollment around the index (no MA-only person-time).
    e = enroll.merge(index[["person_id", "index_date"]], on="person_id")
    e["ok"] = ((e["enroll_start"] <= e["index_date"] - BASE_LO) &
               (e["enroll_end"]   >= e["index_date"] + pd.Timedelta(days=365)) &
               (~e["ma_only"]))
    keep = e.loc[e["ok"], "person_id"].unique()
    df = df[df["person_id"].isin(keep)].copy()
    df["offset"] = (df["submit_date"] - df["index_date"]).dt.days

    # Baseline: submission in [-30, +7]; if several, keep the one closest to index (collapse rule).
    base = df[(df["submit_date"] >= df["index_date"] - BASE_LO) &
              (df["submit_date"] <= df["index_date"] + BASE_HI)].copy()
    base["dist"] = base["offset"].abs()
    base = (base.sort_values(["person_id", "dist"])
                .groupby("person_id").first()
                .reset_index()[["person_id", "total_score"]]
                .rename(columns={"total_score": "baseline_score"}))

    # Map each submission onto the nearest landmark whose +/-14d window it falls in (one record per person-window).
    rows = []
    for m, d in LANDMARKS.items():
        w = df[(df["submit_date"] >= df["index_date"] + pd.Timedelta(days=d) - WINDOW) &
               (df["submit_date"] <= df["index_date"] + pd.Timedelta(days=d) + WINDOW)].copy()
        w["dist"] = (w["offset"] - d).abs()
        w = w.sort_values(["person_id", "dist"]).groupby("person_id").first().reset_index()
        w["landmark"] = m
        rows.append(w[["person_id", "landmark", "total_score"]])
    panel = pd.concat(rows, ignore_index=True).merge(base, on="person_id")
    panel["change"] = panel["total_score"] - panel["baseline_score"]

    # MID-anchored responder/deterioration at each landmark (symmetric thresholds).
    panel["responder"] = (panel["change"] >= MID).astype(int)
    panel["deteriorated"] = (panel["change"] <= -MID).astype(int)
    return panel

def trajectory_model(panel: pd.DataFrame):
    # Mixed model on change-from-baseline over landmarks (MAR; complete-case as a transparency check only).
    return smf.mixedlm("change ~ C(landmark)", panel, groups=panel["person_id"]).fit(reml=True)

library(data.table)
library(mmrm)

LANDMARKS <- c(`3` = 90, `6` = 182, `9` = 274, `12` = 365)  # month -> days from index
WINDOW <- 14L; BASE_LO <- 30L; BASE_HI <- 7L; MID <- 5

build_pro_panel <- function(pro, index, enroll) {
  setDT(pro); setDT(index); setDT(enroll)
  df <- merge(pro, index[, .(person_id, index_date)], by = "person_id")

  e  <- merge(enroll, index[, .(person_id, index_date)], by = "person_id")
  keep <- e[enroll_start <= index_date - BASE_LO &
            enroll_end   >= index_date + 365 & !ma_only, unique(person_id)]
  df <- df[person_id %chin% keep]
  df[, offset := as.integer(submit_date - index_date)]

  base <- df[submit_date >= index_date - BASE_LO & submit_date <= index_date + BASE_HI]
  base <- base[order(person_id, abs(offset))][, .SD[1L], by = person_id][
               , .(person_id, baseline_score = total_score)]

  pick <- rbindlist(lapply(names(LANDMARKS), function(m) {
    d <- LANDMARKS[[m]]
    w <- df[submit_date >= index_date + d - WINDOW & submit_date <= index_date + d + WINDOW]
    w <- w[order(person_id, abs(offset - d))][, .SD[1L], by = person_id]
    w[, .(person_id, landmark = as.integer(m), total_score)]
  }))
  panel <- merge(pick, base, by = "person_id")
  panel[, change := total_score - baseline_score]
  panel[, responder := as.integer(change >= MID)]
  panel[, deteriorated := as.integer(change <= -MID)]
  panel[, landmark := factor(landmark)]
  panel[]
}

# True MMRM: unstructured covariance across landmarks within person, MAR.
fit_mmrm <- function(panel) {
  panel <- as.data.frame(panel)
  mmrm(change ~ landmark + us(landmark | person_id), data = panel)
}