Patient-Reported Outcomes in Real-World Settings

A real-world PRO is a self-reported measure of a patient's health status — symptoms, function, or health-related quality of life — captured outside a protocol-driven trial: in a disease registry, an ePRO app or patient portal, or embedded in EHR workflows (e.g., PROMIS computer-adaptive tests pushed before oncology visits). The instrument may be identical to the one used in a trial; what changes is the data-generating process.

In a trial, assessment timing is scheduled, completion is chased by coordinators, and the analysis population is fixed at randomization. In the real world, assessment is visit-driven or opt-in, completion is voluntary and self-selected, and the patients who keep responding differ systematically from those who stop — usually because the sickest patients are the ones who drop out.

This entry is about the measurement-and-analysis engineering required to turn that messy stream of submissions into a defensible analytic dataset; it is distinct from instrument creation (`pro-development`) and psychometric validation (`pro-validation`), which assume the instrument already performs and ask a different question.

Core conceptual distinction

. Three things separate real-world PRO measurement from trial PRO measurement, and each drives a specific analytic decision. (1) Assessment timing is endogenous. A real-world PRO is recorded when the patient shows up or chooses to respond, not on a fixed schedule, so you must impose an explicit completion window (e.g., a scheduled landmark ±14 days) and a time-zero anchor (initiation, diagnosis, or surgery date — often taken from claims, not from the PRO system) to make scores from different patients comparable.

(2) Missingness is informative. Non-response in routine care is almost never missing-completely-at-random: it correlates with the outcome itself (a patient too sick or too symptomatic to complete a questionnaire), making naive complete-case or last-observation-carried-forward (LOCF) analyses biased toward a falsely stable or falsely improving trajectory.

(3) The cohort is a completer cohort. The patients with usable PRO data are a self-selected subset of the enrolled or treated population, so the estimand silently shifts from "the PRO trajectory of treated patients" to "the PRO trajectory of patients who keep filling out questionnaires." Stating which of these you mean — and whether you are estimating a within-person change, a between-group comparison, or a responder proportion — is the estimand discipline this concept demands.

Pros, cons, and trade-offs

.

• vs trial-collected PROs: Real-world PROs deliver longer follow-up, larger and more representative populations, and capture of subgroups trials exclude (frail, multimorbid, very old). Cost: timing is irregular, completion is low and selective, and there is no coordinator chasing missing forms — so the missing-data and selection problems that are nuisances in a trial become the central methodological threat. Prefer real-world PROs when the question is durability, real-world burden, or HTA value evidence; prefer trial PROs when you need an unbiased treatment-effect estimate on a PRO endpoint.
• vs clinician-reported or claims-derived outcomes: PROs capture the patient's lived experience (fatigue, pain, role function) that claims and chart abstraction systematically miss. Cost: they are subjective, vulnerable to response shift and recall, and far more missing than a claims-based event (an inpatient claim exists whether or not the patient feels like reporting). Prefer PROs for symptom/QoL endpoints; pair them with claims-confirmed clinical events (`endpoint-adjudication-chart-review-rwe`) so that the absence of a PRO is not mistaken for the absence of an event.
• vs complete-case analysis of real-world PROs: Mixed-effects/MMRM models (`mmrm-repeated-measures-rwe`) under a missing-at-random assumption, with multiple imputation (`multiple-imputation-longitudinal-rwe`) and explicit MNAR sensitivity analysis, recover far less biased trajectories than complete-case or LOCF. Cost: more assumptions and code, and MAR is still an assumption that informative dropout can violate. Prefer the model-based approach; reserve complete-case only as a transparency check, never as the primary analysis.

When NOT to use — and when it is actively misleading or dangerous

.

• As an unadjusted treatment-effect estimate. Comparing PRO change between two real-world treatment groups without confounding control and differential-missingness handling is doubly biased: by indication and by who keeps responding. A drug that makes patients feel worse can look better simply because the patients it harms stop completing surveys.
• With LOCF or complete-case as the primary analysis when dropout is informative. LOCF freezes the last (often pre-decline) score and manufactures stability; complete-case conditions on survival-to-response. Both are dangerous in progressive disease (oncology, ALS, advanced heart failure) where the patients who disappear are the ones declining.
• When the completer cohort no longer maps to a meaningful population. If only 20% of an opt-in ePRO cohort responds at month 12 and responders are younger, English-speaking, and higher-SES, the surviving estimand describes a population no decision-maker cares about — report the completion funnel, do not bury it.
• Across instruments or versions without a crosswalk. Pooling SF-36 with PROMIS, or PRO-CTCAE free-text with graded items, without a validated linkage produces a trajectory artifact of the instrument switch, not the patient.
• When the "real-world" PRO is actually a single cross-sectional snapshot marketed as longitudinal evidence — a one-time portal survey cannot support a trajectory or responder claim.

Data-source operational depth

.

• Disease/product registry (scheduled PRO at fixed visits): The best structure for comparability, but completion falls off with disease severity and at later visits, so missingness is informative; LOCF here hides genuine decline. Failure mode: registries often record only the score and visit, not why a form is missing (death? too sick? lost to follow-up?), so you cannot distinguish MAR from MNAR without linking to a mortality source (`mortality-source-hierarchy-rwe`) and claims for hospitalizations. Workaround: build an explicit assessment-expected vs assessment-observed table per visit and model dropout jointly or via MNAR sensitivity (delta-adjusted multiple imputation).
• ePRO app / patient portal (opt-in, push-notification driven): Subject to a digital-divide selection — older, lower-SES, non-English, and less digitally literate patients under-respond — so the responder cohort is not the enrolled cohort. Push-notification timing and reminder cadence change who responds and when, contaminating the completion window. Failure mode: multiple submissions inside one window (a patient answers twice) and unscheduled "I feel awful today" submissions that oversample bad days. Workaround: pre-specify a window-collapse rule (e.g., the submission closest to the scheduled landmark; or first-in-window), cap one record per person-window, and report completion by demographic strata to expose the selection.
• EHR-embedded PROMIS via portal (visit-driven): Captured only when the patient has a visit and the workflow fires, so there is no PRO when the patient leaves the system or is hospitalized elsewhere — visit-driven capture means departure is differential. Failure mode: a "stable" PRO trajectory can mask a claims-confirmed hospitalization that occurred between two portal completions. Workaround: link to claims/EHR encounter data so that clinical events between PRO assessments are observed, and treat a PRO gap that brackets a major event as informative, not ignorable.
• Linked claims + PRO: The strongest substrate — claims supply the time-zero anchor (initiation/surgery date), confirm clinical events and death (so absence of a PRO is not read as absence of an event), and let you characterize non-responders on observable claims covariates to test the MAR assumption. Cost: linkage selection (only the linkable subset has both), and date reconciliation between the PRO timestamp and the claim service date before window assignment.

Worked example (linked registry ePRO + claims)

Question: the 12-month FACT-G total-score trajectory and 6-month responder rate after initiating a new oral oncolytic, among adults in a cancer registry with linked commercial + Medicare FFS claims. (1) Time zero: the first dispensing of the oncolytic from the pharmacy claim (`fill_date` on the index NDC), not the registry enrollment date, so all patients are aligned at treatment initiation.

(2) Eligibility / observability: require 365 days of continuous medical + pharmacy enrollment before `index_date` (exclude Medicare Advantage-only person-time, which lacks FFS claims and would make the baseline window unobservable) and at least one PRO submission in the baseline window. (3) Baseline PRO: the FACT-G submission in the window [`index_date` − 30, `index_date` + 7]; if multiple submissions fall in-window, keep the one closest to `index_date` (window-collapse rule) and cap one record per person-window.

(4) Scheduled landmarks: months 3, 6, 9, 12, each with a completion window of the scheduled day ±14 days; the same collapse and cap rules apply. (5) Responder definition: a patient is a 6-month responder if their month-6 FACT-G is ≥ their baseline plus the MID of 5 points (anchor-based MID; report deterioration symmetrically as a drop ≥5).

(6) Informative missingness: before scoring, build an expected-vs-observed completion table by landmark and by age/SES/language stratum; classify each missing assessment using linked claims and a mortality source as death, hospitalization-near-window, disenrollment, or unexplained — because a patient who is hospitalized (claims-confirmed) at month 6 but has no PRO is informatively, not ignorably, missing.

(7) Primary analysis: an MMRM on change-from-baseline with an unstructured covariance over the four landmarks (MAR), adjusting for baseline score and confounders measured in the 365-day claims lookback; the responder rate is reported under multiple imputation, never LOCF. (8) Sensitivity: delta-adjusted (MNAR) multiple imputation that penalizes imputed scores for dropouts, and a tabulated completion funnel so reviewers see exactly how the analytic cohort narrowed from enrolled to baseline-observed to month-12-observed.

Interpreting the output

Consider the worked example: mean PROMIS Physical Function change from baseline is 1.5 points at 6 months (sum of changes 5 + 2 − 6 + 5 = 6, divided by 4 patients), with 2 of 4 patients (50%) classified as responders (change ≥ 3 points, the MID) and 1 of 4 (25%) classified as deteriorators.

Formal interpretation: A mean change of 1.5 points is below the anchor-based MID of 3 points, meaning that on average the group did not achieve a patient-meaningful improvement — even though the arithmetic mean is positive. The mean and the responder rate tell complementary stories that neither tells alone: the mean of 1.5 masks that two patients improved by 5 points each while one deteriorated by 6.

In routine-care PRO data, missing-not-at-random missingness is the dominant analytic threat: patients who are more symptomatic at month 6 are less likely to complete the questionnaire. If the 1.5-point mean was computed only among completers, it is likely biased upward — the deteriorating patient (P003, change −6) might not have submitted if the context were a real-world registry rather than a controlled example. Reporting completers-only estimates without a missingness model is therefore not interpretable as a population-level treatment effect.

Practical interpretation: Pair the mean change with the completion funnel (number enrolled, number with baseline PRO, number with 6-month PRO) and the MNAR sensitivity analysis results. If the delta-adjusted imputation moves the mean from 1.5 toward a smaller or negative number, the missingness is informative enough to change the clinical conclusion. Report both the responder rate and the deterioration rate: a 50% responder rate with a 25% deterioration rate conveys that the treatment's average effect hides substantial individual-level heterogeneity — a finding relevant to patient communication and subgroup analyses.

Decision diagram

flowchart TD
  Src[Enrolled / treated population] --> Anchor[Time zero from claims<br/>e.g. first oncolytic fill_date]
  Anchor --> Cap["ePRO / portal / registry submissions<br/>irregular, voluntary, often missing"]
  Cap --> Win["Completion windows<br/>baseline [-30,+7]; landmarks +/-14d"]
  Win --> Collapse["Collapse multiple in-window submissions<br/>closest-to-landmark; cap 1 per person-window"]
  Collapse --> Panel["Analytic panel: change-from-baseline<br/>+ MID-anchored responder/deterioration flags"]
  Panel --> Miss["Classify each missing assessment via linked claims + mortality<br/>death / hospitalization / disenroll / unexplained"]
  Miss --> Model["Primary: MMRM under MAR + multiple imputation"]
  Model --> Sens["Sensitivity: delta-adjusted MNAR imputation<br/>+ completion funnel by demographic stratum"]

flowchart TD
  Q{What is the dominant missingness pattern?} -->|Low, plausibly MAR<br/>given observed covariates| M1[MMRM / mixed-effects<br/>under MAR + multiple imputation]
  Q -->|Dropout driven by current unobserved score<br/>informative MNAR| M2[Pattern-mixture / delta-adjusted MI<br/>or selection model]
  Q -->|Dropout tied to a terminal event<br/>death / progression| M3[Joint longitudinal-survival model<br/>or composite responder w/ death = non-responder]
  M1 --> Never[Do NOT use LOCF or complete-case as primary<br/>when dropout is informative]
  M2 --> Never
  M3 --> Never

Worked example

Scenario

Four patients with a chronic inflammatory joint disease start a new oral treatment. Before treatment (baseline) and six months later, each patient completes the PROMIS Physical Function questionnaire, which scores physical ability from 0 to 100 (higher = better). The MID for this scale is 3 points: a change of at least +3 counts as meaningful improvement, and a change of -3 or worse counts as meaningful deterioration. We want to know the mean change in the group and how many patients were responders.

Dataset

PROMIS Physical Function T-scores for four patients at baseline and 6 months. A researcher would see one row per patient per time point in a registry or EHR export.

Steps

Result

Mean change from baseline = 6 / 4 = 1.5 points. This is below the MID of 3 points, so the average group change is not clinically meaningful. Responder rate = 2 / 4 = 50% (patients P001 and P004). Deterioration rate = 1 / 4 = 25% (patient P003).

Trade-offs

Runnable example

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf

LANDMARKS = {3: 90, 6: 182, 9: 274, 12: 365}   # month -> days from index
WINDOW = pd.Timedelta(days=14)                  # +/- completion window around each landmark
BASE_LO, BASE_HI = pd.Timedelta(days=30), pd.Timedelta(days=7)   # baseline window [-30, +7]
MID = 5.0                                        # FACT-G anchor-based minimally important difference

def build_pro_panel(pro: pd.DataFrame, index: pd.DataFrame, enroll: pd.DataFrame) -> pd.DataFrame:
    df = pro.merge(index[["person_id", "index_date"]], on="person_id")

    # Observability: continuous, FFS-observable enrollment around the index (no MA-only person-time).
    e = enroll.merge(index[["person_id", "index_date"]], on="person_id")
    e["ok"] = ((e["enroll_start"] <= e["index_date"] - BASE_LO) &
               (e["enroll_end"]   >= e["index_date"] + pd.Timedelta(days=365)) &
               (~e["ma_only"]))
    keep = e.loc[e["ok"], "person_id"].unique()
    df = df[df["person_id"].isin(keep)].copy()
    df["offset"] = (df["submit_date"] - df["index_date"]).dt.days

    # Baseline: submission in [-30, +7]; if several, keep the one closest to index (collapse rule).
    base = df[(df["submit_date"] >= df["index_date"] - BASE_LO) &
              (df["submit_date"] <= df["index_date"] + BASE_HI)].copy()
    base["dist"] = base["offset"].abs()
    base = (base.sort_values(["person_id", "dist"])
                .groupby("person_id").first()
                .reset_index()[["person_id", "total_score"]]
                .rename(columns={"total_score": "baseline_score"}))

    # Map each submission onto the nearest landmark whose +/-14d window it falls in (one record per person-window).
    rows = []
    for m, d in LANDMARKS.items():
        w = df[(df["submit_date"] >= df["index_date"] + pd.Timedelta(days=d) - WINDOW) &
               (df["submit_date"] <= df["index_date"] + pd.Timedelta(days=d) + WINDOW)].copy()
        w["dist"] = (w["offset"] - d).abs()
        w = w.sort_values(["person_id", "dist"]).groupby("person_id").first().reset_index()
        w["landmark"] = m
        rows.append(w[["person_id", "landmark", "total_score"]])
    panel = pd.concat(rows, ignore_index=True).merge(base, on="person_id")
    panel["change"] = panel["total_score"] - panel["baseline_score"]

    # MID-anchored responder/deterioration at each landmark (symmetric thresholds).
    panel["responder"] = (panel["change"] >= MID).astype(int)
    panel["deteriorated"] = (panel["change"] <= -MID).astype(int)
    return panel

def trajectory_model(panel: pd.DataFrame):
    # Mixed model on change-from-baseline over landmarks (MAR; complete-case as a transparency check only).
    return smf.mixedlm("change ~ C(landmark)", panel, groups=panel["person_id"]).fit(reml=True)

library(data.table)
library(mmrm)

LANDMARKS <- c(`3` = 90, `6` = 182, `9` = 274, `12` = 365)  # month -> days from index
WINDOW <- 14L; BASE_LO <- 30L; BASE_HI <- 7L; MID <- 5

build_pro_panel <- function(pro, index, enroll) {
  setDT(pro); setDT(index); setDT(enroll)
  df <- merge(pro, index[, .(person_id, index_date)], by = "person_id")

  e  <- merge(enroll, index[, .(person_id, index_date)], by = "person_id")
  keep <- e[enroll_start <= index_date - BASE_LO &
            enroll_end   >= index_date + 365 & !ma_only, unique(person_id)]
  df <- df[person_id %chin% keep]
  df[, offset := as.integer(submit_date - index_date)]

  base <- df[submit_date >= index_date - BASE_LO & submit_date <= index_date + BASE_HI]
  base <- base[order(person_id, abs(offset))][, .SD[1L], by = person_id][
               , .(person_id, baseline_score = total_score)]

  pick <- rbindlist(lapply(names(LANDMARKS), function(m) {
    d <- LANDMARKS[[m]]
    w <- df[submit_date >= index_date + d - WINDOW & submit_date <= index_date + d + WINDOW]
    w <- w[order(person_id, abs(offset - d))][, .SD[1L], by = person_id]
    w[, .(person_id, landmark = as.integer(m), total_score)]
  }))
  panel <- merge(pick, base, by = "person_id")
  panel[, change := total_score - baseline_score]
  panel[, responder := as.integer(change >= MID)]
  panel[, deteriorated := as.integer(change <= -MID)]
  panel[, landmark := factor(landmark)]
  panel[]
}

# True MMRM: unstructured covariance across landmarks within person, MAR.
fit_mmrm <- function(panel) {
  panel <- as.data.frame(panel)
  mmrm(change ~ landmark + us(landmark | person_id), data = panel)
}

%let window = 14; %let base_lo = 30; %let base_hi = 7; %let mid = 5;

/* Observable cohort: continuous FFS enrollment around index, no MA-only person-time. */
proc sql;
  create table cohort as
  select distinct i.person_id, i.index_date
  from work.index i
  where exists (select 1 from work.enroll e
                where e.person_id = i.person_id and e.ma_only = 0
                  and e.enroll_start <= i.index_date - &base_lo
                  and e.enroll_end   >= i.index_date + 365);
quit;

/* Baseline PRO in [-30,+7]; keep the submission closest to index (collapse rule). */
proc sql;
  create table base as
  select p.person_id, p.total_score as baseline_score
  from work.pro p inner join cohort c on p.person_id = c.person_id
  where p.submit_date between c.index_date - &base_lo and c.index_date + &base_hi
  group by p.person_id
  having abs(p.submit_date - c.index_date) = min(abs(p.submit_date - c.index_date));
quit;

/* Map submissions to landmarks (months 3/6/9/12) within +/-14d, one record per person-window. */
%macro landmark(m, d);
  proc sql;
    create table lm_&m as
    select p.person_id, &m as landmark, p.total_score
    from work.pro p inner join cohort c on p.person_id = c.person_id
    where p.submit_date between c.index_date + &d - &window and c.index_date + &d + &window
    group by p.person_id
    having abs(p.submit_date - (c.index_date + &d)) =
           min(abs(p.submit_date - (c.index_date + &d)));
  quit;
%mend;
%landmark(3,90); %landmark(6,182); %landmark(9,274); %landmark(12,365);

data panel;
  set lm_3 lm_6 lm_9 lm_12;
run;
proc sql;
  create table panel as
  select a.person_id, a.landmark, a.total_score, b.baseline_score,
         a.total_score - b.baseline_score as change
  from panel a inner join base b on a.person_id = b.person_id;
quit;

/* MID-anchored 6-month responder rate (report under MI, not LOCF). */
data resp; set panel; if landmark = 6; responder = (change >= &mid); run;
proc freq data=resp; tables responder / binomial(level='1'); run;

/* MMRM trajectory: unstructured within-person covariance over landmarks, MAR. */
proc mixed data=panel method=reml;
  class person_id landmark;
  model change = landmark / solution ddfm=kr;
  repeated landmark / subject=person_id type=un;
  lsmeans landmark / cl;
run;

/* Informative-missingness sensitivity: multiply impute, analyze, and pool (delta-adjust for MNAR offline). */
proc mi data=panel out=imp nimpute=20 seed=20240601;
  class landmark;
  var baseline_score change;
run;
proc mixed data=imp method=reml; by _imputation_;
  class person_id landmark;
  model change = landmark / solution; repeated landmark / subject=person_id type=un;
  ods output solutionf=mixparms;
run;
proc mianalyze parms=mixparms; modeleffects landmark; run;

Citations

1. [1]Snyder CF, Aaronson NK, Choucair AK, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Quality of Life Research. 2012;21(8):1305-1314.
2. [2]Greenhalgh J, Gooding K, Gibbons E, et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. Journal of Patient-Reported Outcomes. 2018;2:42.

1. [3]Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. Journal of Clinical Oncology. 2016;34(6):557-565.
2. [4]Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.

1. [5]Mercieca-Bebber R, King MT, Calvert MJ, Stockler MR, Friedlander M. The importance of patient-reported outcomes in clinical trials and strategies for future optimization. Patient Related Outcome Measures. 2018;9:353-367.