Estimands (ATE/ATT) and Intercurrent Events in RWE

In plain language

An estimand is the precise question a study is designed to answer — it forces you to say exactly which patients you are studying, what treatment comparison you are making, and how you will handle events that happen after treatment starts (like a patient switching drugs or dying). Researchers distinguish two main targets: the average treatment effect (ATE) asks what would happen if every eligible patient in the population received the drug versus the comparator, while the average treatment effect in the treated (ATT) asks the narrower question of what happens among the patients who actually chose that drug. Pinning down the estimand before any analysis prevents two analysts from unknowingly answering different questions with the same dataset.

An estimand is the exact causal question an analysis answers, fixed before any estimator is chosen. The ICH E9(R1) addendum (2019) requires five attributes to be specified jointly: (1) population, (2) treatment (intervention) strategy, (3) endpoint, (4) the strategy for each intercurrent event (ICE), and (5) the population-level summary measure (risk difference, risk ratio, hazard ratio, restricted mean survival time difference). In RWE this discipline matters more than in trials, because the ICEs that E9(R1) was written for — treatment discontinuation, switching, dose change, rescue therapy, death, loss of plan enrollment — occur constantly in claims and EHR data and are routinely caused by the same patient factors that drive the outcome. Two analysts running "the same study" on the same database can produce materially different hazard ratios solely because they made different, usually implicit, ICE choices.

Core estimand distinction

Two axes must be pinned down and are independent. (1) Which population? The ATE is E[Y(1) − Y(0)] over the whole eligible population; the ATT is E[Y(1) − Y(0) | A=1], the effect among those who actually initiated the index drug; the ATC is the symmetric quantity in the comparator group. These coincide only when the effect is homogeneous across treated and untreated — almost never true under channeling. The estimator silently encodes the choice: 1:1 PS matching and SMR weighting target the ATT; stabilized IPTW, g-computation standardized over the full cohort, TMLE, and overlap weighting (which targets the ATO, a re-weighted overlap population) target ATE-type quantities. (2) Which ICE strategy? E9(R1) names five — treatment policy (ignore the ICE, follow everyone regardless, the ITT analogue), while-on-treatment (attribute outcomes only while exposed; censor at the ICE), hypothetical (the effect had the ICE not occurred; requires g-methods or IPCW because naive censoring is informative), composite (the ICE is folded into an unfavorable endpoint, e.g. discontinuation-or-death = failure), and principal stratum (the effect in the latent subset who would adhere). The estimand is the question; the g-method, weighting, or clone-censor-weight scheme is the answer that targets it. Reporting a hazard ratio without naming the population and the ICE strategy is, under E9(R1), an incomplete result.

Pros, cons, and trade-offs

(specific and comparative, naming the alternatives). - vs target-trial-emulation: The estimand framework is the specification step that target-trial emulation operationalizes — each E9(R1) ICE strategy maps to a concrete cloning/censoring/weighting rule in the emulated protocol. Cost: the estimand is only the question; you still need a valid identification strategy and exchangeability. Always write the estimand before choosing an estimator; the two are complementary, not alternatives. - vs cox-ph-regression as conventionally reported: Estimand thinking forces clarity on whether a reported HR is marginal (ATE/ATT) or conditional, and which ICE strategy its censoring rules imply. A vanilla `coxph` HR with patients censored at discontinuation is a while-on-treatment, conditional estimand whether or not the analyst intended it. Cost: targeting a marginal ATE/ATT requires standardization or weighting on top of Cox, not just the model coefficient; under non-proportional hazards the HR is not a clean summary at all and RMST or risk differences are preferred. - vs g-estimation-structural-nested-models: G-estimation is the natural engine for hypothetical and dynamic-regime estimands under time-varying ICEs and time-varying confounding. Cost: it is harder to specify, communicate to reviewers, and debug than a treatment-policy ATE from IPTW or matching. Prefer plain treatment-policy estimands unless the decision genuinely hinges on "what if patients had stayed on therapy." - vs an unspecified/default analysis: Naming the estimand exposes mismatches that otherwise hide — most commonly reporting an ATT (from matching) when the policy question is population-level (a formulary or coverage decision for all eligible patients), which is an ATE question. The structured estimand is more work upfront and pays for itself at review.

When to use

(clear decision rules). Always, as the first step of any comparative RWE protocol or SAP, and especially when: ICEs are frequent and prognostic (oncology switching at progression, adherence-driven discontinuation in chronic disease); the same dataset will support multiple analyses that must be distinguished; the study informs a regulatory or HTA decision where E9(R1) language is now expected; or treatment effects are heterogeneous so ATE and ATT diverge and the audience needs to know which they are getting.

When NOT to use — and when it is actively misleading or dangerous

(clear decision rules). The estimand framework is never inappropriate to apply, but specific choices within it are dangerous. (1) A hypothetical strategy when the ICE process is driven by unmeasured confounders is the classic trap: censoring at discontinuation (or weighting for it) assumes no unmeasured common cause of discontinuation and outcome. When sick patients stop therapy because they are deteriorating, the while-on-treatment / hypothetical estimand is biased toward benefit — the analysis manufactures efficacy. Diagnose with the plausibility of the no-unmeasured-confounding-for-censoring assumption, not with model fit. (2) Reporting an ATT for a population-level policy question systematically answers the wrong question; if one drug is reserved for a sicker subgroup, the ATT describes that subgroup, not the population the decision-maker governs. (3) A while-on-treatment estimand for a long-latency outcome (e.g. cancer incidence) discards exactly the person-time when the outcome occurs and is rarely defensible. (4) Treating death as ordinary censoring in any non-mortality estimand removes a competing event and biases the cause-specific quantity; death must be handled as an ICE (composite, or competing-risk framing). (5) Naming five attributes but then letting the estimator default (e.g. matching) silently override the stated population is worse than no framework, because it provides false assurance.

Data-source operational depth

(claims vs EHR vs registry vs linked). - Claims (FFS vs Medicare Advantage): ICEs are inferred from utilization, never observed directly. Discontinuation is a gap beyond `days_supply` + grace; switching is the first fill of the comparator class; death may come from an inpatient discharge disposition or a linked vital-records/Master Beneficiary Summary file. Failure mode: MA-only person-time lacks FFS claims, so a "discontinuation" can be unobserved capitated care, fabricating an ICE — restrict to enrollees with complete A/B/D (or commercial medical+pharmacy) and exclude MA-only spans before defining any ICE. Differential competing risks by exposure in the elderly mean the drug arm with higher background mortality looks artificially better on a non-fatal endpoint if death is censored rather than treated as an ICE. - EHR: Orders and medication lists overstate true exposure (a stopped drug lingers on the list), so the while-on- treatment and hypothetical estimands are especially fragile; link to dispensing to confirm the ICE. Loss to follow-up is visit-driven and informative — a patient who deteriorates may exit the system, masquerading as administrative censoring and biasing a hypothetical estimand. NLP-extracted discontinuation reasons (toxicity, progression, cost) are needed to choose between composite and hypothetical strategies but are themselves missing-not-at-random. - Registry: Often the cleanest source for endpoint timing, adjudicated death, and reasons for treatment change — ideal for validating claims-based ICE definitions — but typically incomplete for full pharmacy exposure between visits. - Linked claims–EHR–vital records: The ideal substrate (EHR severity + claims completeness + reliable mortality for the death ICE), but linkage selection and order/fill/service-date discrepancies must be reconciled before ICE dates are fixed, or immortal time is introduced (e.g. counting a procedure study's pre-procedure survival toward the treated arm).

Worked claims example (oncology, where ICE strategy decides the answer)

Question: overall survival with first-line BRAF+MEK inhibitor vs anti-PD-1 immunotherapy in adults with metastatic melanoma, in a linked commercial + Medicare FFS database. (1) Eligibility/population: adults with a metastatic-melanoma diagnosis and 365 days of continuous A/B/D (or commercial medical+pharmacy) enrollment before the first qualifying systemic-therapy fill/administration; exclude MA-only person-time so absence of subsequent claims is real. (2) Treatment strategy: initiate the index regimen at `index_date`, arm assigned from the NDC/HCPCS on that date. (3) Endpoint: all-cause death (linked vital records) within 36 months. (4) The decisive ICE — switching at progression: a large fraction of BRAF+MEK initiators switch to immunotherapy at progression. Under treatment policy (follow everyone to death/disenrollment/end-of-data regardless of switch) the contrast describes the initiation strategy and is the policy-relevant estimand for a first-line coverage decision; under a hypothetical strategy (effect had no switch occurred) you must IPCW-censor at the switch fill, modeling the switch hazard from time-varying covariates — and you must defend that no unmeasured factor drives both switching and death. (5) The death ICE: death is the endpoint here, so no special handling; for a non-fatal endpoint (e.g. time-to-next- treatment) death would be a competing-risk ICE, not censoring. (6) Estimator → population: IPTW on the high-dimensional baseline PS standardized over the whole cohort targets the ATE (the formulary question); 1:1 PS matching would instead target the ATT among BRAF+MEK initiators. (7) Reporting: present both treatment-policy and hypothetical estimands as pre-specified primary/secondary — not as a single "sensitivity analysis," because they answer different questions — each with its summary measure (RMST difference at 36 months alongside the HR, since proportional hazards is implausible with a delayed immunotherapy effect).

Interpreting the output

In the Drug A versus Drug B kidney-event study (200 per arm, 12-month follow-up), the analysis reports four estimand-specific results: treatment-policy ATE risk difference = −3.0 percentage points (9.0% vs 12.0%); hypothetical ATE ≈ −3.6 pp; while-on-treatment ATT = −3.6 pp; composite (discontinuation-or-event) RD ≈ +1.0 pp.

(1) Formal interpretation. Each number answers a different causal question. The treatment-policy ATE (−3.0 pp) estimates the effect of assignment to Drug A in the full eligible population, including periods after discontinuation or switching — the ITT analogue. The while-on-treatment ATT (−3.6 pp) estimates the effect among patients who actually initiated Drug A, restricted to time they remained on therapy. The composite estimate (+1.0 pp) is opposite in sign because discontinuations in the Drug A arm — counted as events — outweigh the pharmacological benefit during the observation window. None of these four numbers is the "right" answer; each answers a different policy or regulatory question, and they diverge precisely because Drug A and Drug B initiators differ in adherence behavior and discontinuation rates.

(2) Practical interpretation. A payer asking "what happens to my members assigned Drug A?" needs the treatment-policy ATE. A clinician asking "does Drug A work while the patient takes it?" needs the while-on-treatment ATT. An HTA body assessing adherence-adjusted long-run impact may need the hypothetical estimand under g-computation with IPCW. Reporting only one estimate without naming the estimand and its intercurrent-event strategy creates an interpretability gap that cannot be resolved post hoc.

Worked example

Scenario

A health-plan researcher is comparing a new oral diabetes drug (Drug A) versus metformin (Drug B) on the one-year risk of a serious kidney event. The cohort has 400 new users: 200 started Drug A and 200 started Drug B. After starting, some patients in each arm switch to the other drug or stop entirely — these are the intercurrent events. The researcher must decide: (1) which population defines the effect, and (2) what rule to apply to the switchers.

Dataset

Summary of the 400-patient cohort after one year of follow-up. Each row is one arm; columns show who switched and who had the kidney event.

Steps

• Step 1 — Choose the population (ATE vs ATT). For ATE, the question is: what would happen to all 400 patients if everyone took Drug A vs if everyone took Drug B? For ATT, the question is narrower: what would have happened to the 200 Drug A initiators specifically if they had taken Drug B instead?

• Step 2 — Choose the intercurrent-event strategy. The four options from ICH E9(R1) are: (a) Treatment policy — count every patient's outcome regardless of switching (like ITT); (b) Hypothetical — estimate the outcome as if no one had switched (requires special weighting methods); (c) While-on-treatment — count only the outcome that occurred before switching, ignore the rest; (d) Composite — treat switching as itself a bad outcome, so switching + kidney event both count as failures.

• Step 3 — Apply treatment policy (the simplest and most common strategy). Drug A arm: 12 + 6 = 18 kidney events out of 200 patients = 9.0% risk. Drug B arm: 20 + 4 = 24 kidney events out of 200 patients = 12.0% risk. Treatment-policy ATE risk difference = 9.0% minus 12.0% = -3.0 percentage points (Drug A lower).

• Step 4 — Contrast with the ATT. The ATT focuses on the 200 Drug A initiators only. Under a hypothetical estimand (had no one switched), we attribute the 12 events among the 160 stayers but must estimate what the 40 switchers would have experienced on Drug A. Using the stayer rate as a proxy: 40 x 0.075 = 3 projected events. Drug A risk = (12 + 3) / 200 = 7.5%. Counterfactual Drug B risk for these same 200 patients (using Drug B stayer rate) = (200 x 0.111) = 22.2 projected events = 11.1%. ATT risk difference = 7.5% minus 11.1% = -3.6 percentage points — slightly larger than ATE because Drug A initiators happen to be lower-risk patients who also respond better.

• Step 5 — Recognize what each estimand answers. ATE (-3.0 pp) answers the formulary question: should the health plan cover Drug A for all eligible diabetic members? ATT (-3.6 pp) answers the effectiveness question: for patients who chose Drug A, did it help them? Different questions, different numbers, same dataset.

Result

Chosen estimand: treatment-policy ATE, because the study informs a population-level coverage decision. Risk difference = -3.0 percentage points (9.0% vs 12.0%), meaning Drug A is associated with 3 fewer kidney events per 100 patients over one year when every initiator is followed regardless of switching. The ATT would have given -3.6 pp, which is a subtly different answer to a subtly different question — and reporting the wrong one for the policy decision would overstate or understate the benefit for the intended audience.

Ice Strategy Contrast

The four intercurrent-event strategies applied to the same 400-patient cohort. Each strategy produces a numerically and conceptually different answer.

Runnable example

import numpy as np
import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf

X = ["X1", "X2", "X3"]  # replace with the full baseline confounder set

# ---- Block 1: treatment-policy ATE and ATT via g-computation (standardization) ----
# Fit an outcome model that includes treatment A and confounders, then standardize
# predictions setting A=1 and A=0 over the chosen population (whole cohort -> ATE; A==1 -> ATT).
def gcomp_ate_att(df, outcome="Y", treat="A", covars=X):
    f = f"{outcome} ~ {treat} + " + " + ".join(covars)
    m = smf.glm(f, data=df, family=sm.families.Binomial()).fit()
    d1, d0 = df.copy(), df.copy()
    d1[treat], d0[treat] = 1, 0
    p1, p0 = m.predict(d1), m.predict(d0)
    ate = (p1 - p0).mean()                      # population: whole cohort
    treated = df[treat] == 1
    att = (p1[treated] - p0[treated]).mean()    # population: the treated
    return ate, att

def boot_ci(df, fn, B=1000, seed=1):
    rng = np.random.default_rng(seed)
    idx = np.arange(len(df))
    est = np.array([fn(df.iloc[rng.choice(idx, len(df), replace=True)]) for _ in range(B)])
    return est.mean(axis=0), np.percentile(est, [2.5, 97.5], axis=0)

ate, att = gcomp_ate_att(df)
(m_ate_att, ci) = boot_ci(df, lambda d: np.array(gcomp_ate_att(d)))
print(f"Treatment-policy ATE risk diff = {ate:+.3f}  95% CI [{ci[0,0]:+.3f}, {ci[1,0]:+.3f}]")
print(f"Treatment-policy ATT risk diff = {att:+.3f}  95% CI [{ci[0,1]:+.3f}, {ci[1,1]:+.3f}]")

# ---- Block 2: hypothetical (had-no-switch) estimand via stabilized IPCW ----
# Expand to person-interval (long) format; censor at the ICE (discontinue_t) and re-weight
# so the censored intervals stand in for the population that would have continued.
# Requires time-varying covariates Lt; here baseline X are carried forward as a minimal example.
def to_long(df, interval=30):
    rows = []
    for _, r in df.iterrows():
        stop = int(np.ceil(r.time_to_event / interval))
        ice = r.discontinue_t if not pd.isna(r.discontinue_t) else np.inf
        for k in range(stop):
            t0, t1 = k * interval, min((k + 1) * interval, r.time_to_event)
            censored = (ice <= t1)                      # ICE during this interval -> artificially censored
            event = int(r.event and (t1 >= r.time_to_event) and not censored)
            rows.append({**r[["person_id", "A"] + X].to_dict(),
                         "k": k, "event": event, "cens": int(censored)})
            if censored:
                break
    return pd.DataFrame(rows)

lng = to_long(df)
# Stabilized inverse-probability-of-censoring weights from the censoring (ICE) hazard.
cm_num = smf.glm("cens ~ A + k", data=lng, family=sm.families.Binomial()).fit()
cm_den = smf.glm("cens ~ A + k + " + " + ".join(X), data=lng,
                 family=sm.families.Binomial()).fit()
p_uncens_num = 1 - cm_num.predict(lng)
p_uncens_den = 1 - cm_den.predict(lng)
lng["sw"] = p_uncens_num / p_uncens_den
lng["sw"] = lng.groupby("person_id")["sw"].cumprod().clip(upper=20)  # cumulative, truncate extremes
# Weighted pooled-logistic hazard model -> hypothetical (no-ICE) risk by standardization.
hz = smf.glm("event ~ A + k + " + " + ".join(X), data=lng,
             family=sm.families.Binomial(), freq_weights=lng["sw"]).fit()
h1, h0 = lng.copy(), lng.copy(); h1["A"], h0["A"] = 1, 0
risk1 = 1 - (1 - hz.predict(h1)).groupby(lng["person_id"]).prod().mean()
risk0 = 1 - (1 - hz.predict(h0)).groupby(lng["person_id"]).prod().mean()
print(f"Hypothetical (no-ICE) ATE risk diff = {risk1 - risk0:+.3f}")

library(marginaleffects)
library(ipw)
library(survival)

Xrhs <- "X1 + X2 + X3"  # replace with the full baseline confounder set

## ---- Block 1: treatment-policy ATE and ATT via g-computation ----
fit <- glm(as.formula(paste("Y ~ A +", Xrhs)), family = binomial, data = df)

ate <- avg_comparisons(fit, variables = "A")                       # population: whole cohort
att <- avg_comparisons(fit, variables = "A",
                       newdata = subset(df, A == 1))               # population: the treated
print(ate); print(att)

## ---- Block 2: hypothetical (had-no-switch) estimand via stabilized IPCW ----
## Expand to person-interval long format, with a censoring indicator at the ICE time.
long <- survSplit(Surv(time_to_event, event) ~ ., data = df,
                  cut = seq(30, max(df$time_to_event), by = 30), episode = "k")
long$cens <- with(long, !is.na(discontinue_t) & discontinue_t <= time_to_event)

## Stabilized inverse-probability-of-censoring weights (numerator ~ A only; denominator ~ A + confounders).
w <- ipwtm(exposure = cens, family = "binomial",
           numerator = ~ A,
           denominator = as.formula(paste("~ A +", Xrhs)),
           id = person_id, tstart = tstart, timevar = k,
           type = "first", data = long)
long$ipcw <- pmin(w$ipw.weights, 20)                              # truncate extreme weights

## Weighted Cox targets the hypothetical-strategy hazard ratio (robust SE for the weights).
cox_hyp <- coxph(Surv(tstart, time_to_event, event) ~ A + X1 + X2 + X3,
                 data = long, weights = ipcw, robust = TRUE, id = person_id)
summary(cox_hyp)