Per-Protocol Analysis in RWE and Target Trials

Per-protocol analysis

targets the effect that would be observed if patients followed the treatment protocol specified at time zero. In a trial, that protocol might be "take assigned treatment, avoid prohibited rescue therapy, and attend outcome visits." In an RWE target-trial emulation, the protocol is the explicit treatment strategy: start a drug within a grace period, remain covered by refills, avoid switching, follow a dose-escalation rule, or continue a device/procedure pathway.

The per-protocol estimand is therefore an adherence-specific causal question, not simply an analysis of the subset of people who happened to comply.

The central problem is that adherence is post-baseline behavior. Patients stop, switch, delay, or intensify treatment for reasons related to prognosis, early toxicity, symptom improvement, access, and clinician judgment. Dropping non-adherent patients or censoring at deviation without adjustment selects a healthier, sicker, or otherwise different group and can be more biased than a clean ITT analysis.

A defensible per-protocol analysis treats protocol deviation as an intercurrent event, pre-specifies the deviation rule, and uses g-methods - most commonly inverse-probability-of- censoring weighting (IPCW), marginal structural models, or clone-censor-weight - to recover the target population that would have remained had everyone followed the protocol.

Pros, cons, and trade-offs

• vs intention-to-treat analysis: Per-protocol estimates the effect under adherence, which is often closer to the biological or mechanistic question. It is essential for sustained-treatment, duration, and treat-to-target strategies. Cost: it relies on stronger assumptions than ITT, especially no unmeasured common causes of protocol deviation and outcome, correct censoring/adherence models, and positivity of adherence within covariate histories.
• vs naive per-protocol set analysis: A naive per-protocol set keeps only observed adherers and discards violators. That is easy to program but conditions on a post-baseline variable. Modern per-protocol analysis keeps the target population at time zero and adjusts for deviation-related censoring, making the full-adherence question explicit and auditable.
• vs as-treated analysis: As-treated windows attribute risk while treatment is current; per-protocol analysis asks what would happen under a full strategy, including allowed gaps, switches that trigger deviation, and possibly grace periods. A simple as-treated exposure split can answer acute safety questions but usually does not recover the causal effect of a sustained protocol unless paired with weighting.
• vs clone-censor-weight: Clone-censor-weight is a per-protocol design engine for strategies that are not distinguishable at baseline, such as "initiate within 180 days and persist" vs "never initiate." For two active comparator drugs that are already distinguishable at index fill, ordinary censor-at-deviation plus IPCW is often sufficient; cloning is unnecessary unless the protocol arm is defined by future behavior.

When NOT to use - and when it is actively misleading

• Adherence cannot be observed. If pharmacy fills, administration records, device use, or visit attendance are missing, a protocol-deviation rule cannot fire. Medicare Advantage-only person-time, unlinked EHR orders without dispensing, or registry visits with coarse stop dates can make per-protocol classification look precise while being wrong.
• Deviation is driven by unmeasured prognosis. If patients stop because of early toxicity, worsening disease, response, frailty, or access barriers that the data do not measure, IPCW cannot remove the resulting selection bias. The estimate then has the form of a per-protocol effect but the identifying assumption fails.
• Positivity collapses. If nearly all high-risk patients discontinue, or nearly no frail patients remain adherent, inverse-probability weights explode and the full-adherence estimand is effectively unsupported. Truncation may reduce variance but does not create evidence where no support exists.
• The protocol is a post-hoc description of observed behavior. Per-protocol analysis must start from a protocol that could have been assigned at time zero. "Patients who managed to complete treatment" is not a baseline strategy; it is a selected survivor/adherer group unless encoded by cloning, censoring, and weighting from time zero.
• The policy question is usual-care initiation. If decision makers care about starting a strategy in the real world, including imperfect adherence, ITT should remain primary and per-protocol should be framed as a companion estimand.

Data-source operational depth

- Claims: Protocol adherence usually comes from pharmacy fills: `fill_date + days_supply`, stockpiling/carryover, permissible gaps, switching NDC/classes, disenrollment, and death. Require observable medical + pharmacy benefit through each period; a no-fill gap is only a deviation if fills would have been captured.

The IPCW model should use time-updated utilization, recent hospitalization, new diagnoses, prior adherence, copays or access proxies when available, and baseline covariates. Report weight distribution, truncation, effective sample size, and grace-period sensitivity.

• EHR: Orders and administrations can capture in-clinic treatment precisely and record reasons for deviation (toxicity, response, contraindication), which strengthen the censoring model. But outpatient prescriptions may be ordered but never filled, and patients can receive care outside the system. Link to dispensing or claims for chronic drugs; treat loss of active EHR observation as a separate censoring process.
• Registry: Registries often provide adjudicated outcomes and structured treatment status at visits, but visit-level recording may miss short gaps and outside treatment. Use registry data for disease severity and reasons for deviation, and link to claims/pharmacy/vital records for day-level adherence and competing-risk handling.
• Linked data: Linked claims-EHR-registry-vital records are ideal because they combine fill completeness, clinical reasons for deviation, severity, and death. The linkable subset may be selected, so describe the target population and transportability limits explicitly.

Worked claims example

Question: among new SGLT2i vs DPP-4i initiators with type 2 diabetes and CKD, what is the 2-year per-protocol effect of remaining on the initially assigned class without switching? Time zero is the first qualifying fill after a 365-day washout. The protocol allows refill gaps up to 30 days after supply runs out. A patient deviates if they switch to the other class, fail to refill within the gap rule, disenroll from observable pharmacy coverage, or receive a prohibited add-on defined in the protocol.

Follow-up is artificially censored at the first deviation unless the outcome has already occurred. Fit a pooled logistic censoring model in person-months using baseline and time-updated covariates, form stabilized IPC weights, truncate extreme weights as pre-specified, then fit a weighted discrete-time outcome model and standardize 2-year cumulative incidence by assigned arm. The result is the effect that would have been expected if everyone in the eligible new-user population had remained adherent to their initial class under the observed covariate history assumptions.

Decision diagram

flowchart TD
  T0[Time zero: assign protocol strategy] --> PP[Build person-period adherence status]
  PP --> Dev{Deviation before outcome?}
  Dev -->|No| Keep[Keep person-time and events]
  Dev -->|Yes| Cens[Artificially censor at first deviation]
  Cens --> W[IPCW: reweight still-adherent person-time]
  Keep --> W
  W --> Out[Weighted outcome model]
  Out --> Risk[Standardized full-adherence risk contrast]

Worked example

Scenario

A claims analyst compares new SGLT2i and DPP-4i initiators but wants the sustained-use effect rather than the usual-care initiation effect. The protocol says patients must remain on their index class, may have no refill gap beyond 30 days after supply runs out, and may not switch to the comparator class. The analyst censors person-time at the first protocol deviation and uses IPCW to adjust for the fact that deviation is prognostic.

Dataset

Simplified person-period records for a per-protocol analysis. The deviation month is censored unless the outcome occurred first.

Steps

Result

The per-protocol dataset is a weighted person-period file. Each person starts in the same index arm as the ITT analysis, but follow-up stops at deviation and the remaining adherent person-time is reweighted to represent the original eligible target population under the full-adherence protocol.

Trade-offs

Runnable example

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
import statsmodels.api as sm

def prepare_pp(pp):
    pp = pp.sort_values(["person_id", "month"]).copy()
    # Outcome month is retained even if deviation is also flagged; the event happened before censoring can remove it.
    pp.loc[pp["event"] == 1, "deviated"] = 0
    pp["uncensored"] = 1 - pp["deviated"]
    pp["t"] = pp["month"].astype(float)
    return pp

def stabilized_ipcw(pp, covariates):
    pp = prepare_pp(pp)
    rhs_den = "assigned_arm + t + I(t ** 2) + " + " + ".join(covariates)
    rhs_num = "assigned_arm + t + I(t ** 2)"
    den = smf.logit("uncensored ~ " + rhs_den, data=pp).fit(disp=False)
    num = smf.logit("uncensored ~ " + rhs_num, data=pp).fit(disp=False)
    pp["p_den"] = den.predict(pp).clip(0.01, 0.99)
    pp["p_num"] = num.predict(pp).clip(0.01, 0.99)
    pp["sw"] = (pp.groupby("person_id")["p_num"].cumprod() /
                pp.groupby("person_id")["p_den"].cumprod())
    pp["sw"] = pp["sw"].clip(upper=pp["sw"].quantile(0.99))
    return pp

def fit_per_protocol(pp, covariates):
    w = stabilized_ipcw(pp, covariates)
    analytic = w[w["deviated"] == 0].copy()
    model = smf.glm("event ~ assigned_arm + t + I(t ** 2)",
                    data=analytic,
                    family=sm.families.Binomial(),
                    freq_weights=analytic["sw"]).fit()
    risks = {}
    grid = analytic[["t"]].drop_duplicates().sort_values("t")
    for arm in sorted(analytic["assigned_arm"].unique()):
        pred_grid = grid.assign(assigned_arm=arm)
        hazards = model.predict(pred_grid).to_numpy()
        risks[arm] = float(1 - np.prod(1 - hazards))
    arms = sorted(risks)
    return {"model": model, "risk_by_arm": risks,
            "risk_difference": risks[arms[1]] - risks[arms[0]],
            "weights": w}

library(data.table)

prepare_pp <- function(pp) {
  setDT(pp); setorder(pp, person_id, month)
  pp[event == 1L, deviated := 0L]
  pp[, uncensored := 1L - deviated]
  pp[, t := as.numeric(month)]
  pp[]
}

stabilized_ipcw <- function(pp, covariates) {
  pp <- prepare_pp(copy(pp))
  fden <- as.formula(paste("uncensored ~ assigned_arm + t + I(t^2) +",
                           paste(covariates, collapse = " + ")))
  fnum <- uncensored ~ assigned_arm + t + I(t^2)
  den <- glm(fden, data = pp, family = binomial())
  num <- glm(fnum, data = pp, family = binomial())
  pp[, p_den := pmin(pmax(predict(den, pp, type = "response"), 0.01), 0.99)]
  pp[, p_num := pmin(pmax(predict(num, pp, type = "response"), 0.01), 0.99)]
  pp[, sw := cumprod(p_num) / cumprod(p_den), by = person_id]
  pp[, sw := pmin(sw, quantile(sw, 0.99))]
  pp[]
}

fit_per_protocol <- function(pp, covariates) {
  w <- stabilized_ipcw(pp, covariates)
  analytic <- w[deviated == 0L]
  fit <- glm(event ~ assigned_arm + t + I(t^2), data = analytic,
             family = binomial(), weights = sw)
  grid <- unique(analytic[, .(t)])[order(t)]
  arms <- sort(unique(analytic$assigned_arm))
  risk <- sapply(arms, function(a) {
    h <- predict(fit, cbind(grid, assigned_arm = a), type = "response")
    1 - prod(1 - h)
  })
  list(fit = fit, risk_by_arm = risk,
       risk_difference = unname(risk[2] - risk[1]), weights = w)
}

proc sort data=work.pp; by person_id month; run;

data pp0;
  set work.pp;
  if event = 1 then deviated = 0;       /* retain observed event months */
  uncensored = 1 - deviated;
  t = month;
  t2 = month * month;
run;

proc logistic data=pp0 noprint;
  class assigned_arm(ref='DPP4i') / param=ref;
  model uncensored(event='1') = assigned_arm t t2 age baseline_hosp recent_hosp recent_er prior_adherence;
  output out=pden p=p_den;
run;

proc logistic data=pp0 noprint;
  class assigned_arm(ref='DPP4i') / param=ref;
  model uncensored(event='1') = assigned_arm t t2;
  output out=pnum p=p_num;
run;

proc sort data=pden; by person_id month; run;
proc sort data=pnum; by person_id month; run;

data weights;
  merge pden pnum(keep=person_id month p_num);
  by person_id month;
  retain cum_num cum_den;
  if first.person_id then do; cum_num = 1; cum_den = 1; end;
  p_den = min(max(p_den, 0.01), 0.99);
  p_num = min(max(p_num, 0.01), 0.99);
  cum_num = cum_num * p_num;
  cum_den = cum_den * p_den;
  sw = cum_num / cum_den;
run;

proc univariate data=weights noprint;
  var sw;
  output out=p99 pctlpts=99 pctlpre=p;
run;

data analytic;
  if _n_ = 1 then set p99;
  set weights;
  if sw > p99 then sw = p99;
  if deviated = 0;
run;

proc genmod data=analytic;
  class assigned_arm person_id;
  weight sw;
  model event(event='1') = assigned_arm t t2 / dist=bin link=logit;
  repeated subject=person_id / type=ind;
  estimate 'SGLT2i vs DPP4i per-protocol log odds' assigned_arm 1 -1;
run;

Citations

1. [1]Hernan MA, Robins JM. Per-protocol analyses of pragmatic trials. New England Journal of Medicine. 2017;377(14):1391-1398.
2. [2]International Council for Harmonisation. ICH E9(R1) addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials. EMA scientific guideline page.
3. [3]Hernan MA, Hernandez-Diaz S. Beyond the intention-to-treat in comparative effectiveness research. Clinical Trials. 2012;9(1):48-55.

1. [4]Danaei G, Garcia Rodriguez LA, Cantero OF, Logan RW, Hernan MA. Electronic medical records can be used to emulate target trials of sustained treatment strategies. Journal of Clinical Epidemiology. 2018;96:12-22.

1. [5]Hernan MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology. 2016;183(8):758-764.