Per-Protocol Analysis in RWE and Target Trials

In plain language

Per-protocol analysis asks what would happen if patients actually followed the treatment rule written in the study protocol. In real-world data, that means defining exactly what counts as staying on protocol, such as refilling on time and not switching drugs, then correcting for the fact that the people who stay adherent are usually different from the people who stop or switch.

Per-protocol analysis

targets the effect that would be observed if patients followed the treatment protocol specified at time zero. In a trial, that protocol might be "take assigned treatment, avoid prohibited rescue therapy, and attend outcome visits." In an RWE target-trial emulation, the protocol is the explicit treatment strategy: start a drug within a grace period, remain covered by refills, avoid switching, follow a dose-escalation rule, or continue a device/procedure pathway. The per-protocol estimand is therefore an adherence-specific causal question, not simply an analysis of the subset of people who happened to comply.

The central problem is that adherence is post-baseline behavior. Patients stop, switch, delay, or intensify treatment for reasons related to prognosis, early toxicity, symptom improvement, access, and clinician judgment. Dropping non-adherent patients or censoring at deviation without adjustment selects a healthier, sicker, or otherwise different group and can be more biased than a clean ITT analysis. A defensible per-protocol analysis treats protocol deviation as an intercurrent event, pre-specifies the deviation rule, and uses g-methods - most commonly inverse-probability-of- censoring weighting (IPCW), marginal structural models, or clone-censor-weight - to recover the target population that would have remained had everyone followed the protocol.

Pros, cons, and trade-offs

- vs intention-to-treat analysis: Per-protocol estimates the effect under adherence, which is often closer to the biological or mechanistic question. It is essential for sustained-treatment, duration, and treat-to-target strategies. Cost: it relies on stronger assumptions than ITT, especially no unmeasured common causes of protocol deviation and outcome, correct censoring/adherence models, and positivity of adherence within covariate histories. - vs naive per-protocol set analysis: A naive per-protocol set keeps only observed adherers and discards violators. That is easy to program but conditions on a post-baseline variable. Modern per-protocol analysis keeps the target population at time zero and adjusts for deviation-related censoring, making the full-adherence question explicit and auditable. - vs as-treated analysis: As-treated windows attribute risk while treatment is current; per-protocol analysis asks what would happen under a full strategy, including allowed gaps, switches that trigger deviation, and possibly grace periods. A simple as-treated exposure split can answer acute safety questions but usually does not recover the causal effect of a sustained protocol unless paired with weighting. - vs clone-censor-weight: Clone-censor-weight is a per-protocol design engine for strategies that are not distinguishable at baseline, such as "initiate within 180 days and persist" vs "never initiate." For two active comparator drugs that are already distinguishable at index fill, ordinary censor-at-deviation plus IPCW is often sufficient; cloning is unnecessary unless the protocol arm is defined by future behavior.

When to use

Use per-protocol analysis when the target question is adherence-specific: the efficacy or safety of remaining on a treatment, the effect of a treatment-duration rule, the consequence of following a titration or treat-to- target algorithm, or the sustained-strategy companion to an ITT target-trial emulation. It is most defensible when the data can observe the events that define protocol adherence, time-varying predictors of deviation are measured before each decision point, and there is enough overlap that both continuing and deviating remain possible across covariate histories.

When NOT to use - and when it is actively misleading

- Adherence cannot be observed. If pharmacy fills, administration records, device use, or visit attendance are missing, a protocol-deviation rule cannot fire. Medicare Advantage-only person-time, unlinked EHR orders without dispensing, or registry visits with coarse stop dates can make per-protocol classification look precise while being wrong. - Deviation is driven by unmeasured prognosis. If patients stop because of early toxicity, worsening disease, response, frailty, or access barriers that the data do not measure, IPCW cannot remove the resulting selection bias. The estimate then has the form of a per-protocol effect but the identifying assumption fails. - Positivity collapses. If nearly all high-risk patients discontinue, or nearly no frail patients remain adherent, inverse-probability weights explode and the full-adherence estimand is effectively unsupported. Truncation may reduce variance but does not create evidence where no support exists. - The protocol is a post-hoc description of observed behavior. Per-protocol analysis must start from a protocol that could have been assigned at time zero. "Patients who managed to complete treatment" is not a baseline strategy; it is a selected survivor/adherer group unless encoded by cloning, censoring, and weighting from time zero. - The policy question is usual-care initiation. If decision makers care about starting a strategy in the real world, including imperfect adherence, ITT should remain primary and per-protocol should be framed as a companion estimand.

Data-source operational depth

- Claims: Protocol adherence usually comes from pharmacy fills: `fill_date + days_supply`, stockpiling/carryover, permissible gaps, switching NDC/classes, disenrollment, and death. Require observable medical + pharmacy benefit through each period; a no-fill gap is only a deviation if fills would have been captured. The IPCW model should use time-updated utilization, recent hospitalization, new diagnoses, prior adherence, copays or access proxies when available, and baseline covariates. Report weight distribution, truncation, effective sample size, and grace-period sensitivity. - EHR: Orders and administrations can capture in-clinic treatment precisely and record reasons for deviation (toxicity, response, contraindication), which strengthen the censoring model. But outpatient prescriptions may be ordered but never filled, and patients can receive care outside the system. Link to dispensing or claims for chronic drugs; treat loss of active EHR observation as a separate censoring process. - Registry: Registries often provide adjudicated outcomes and structured treatment status at visits, but visit-level recording may miss short gaps and outside treatment. Use registry data for disease severity and reasons for deviation, and link to claims/pharmacy/vital records for day-level adherence and competing-risk handling. - Linked data: Linked claims-EHR-registry-vital records are ideal because they combine fill completeness, clinical reasons for deviation, severity, and death. The linkable subset may be selected, so describe the target population and transportability limits explicitly.

Worked claims example

Question: among new SGLT2i vs DPP-4i initiators with type 2 diabetes and CKD, what is the 2-year per-protocol effect of remaining on the initially assigned class without switching? Time zero is the first qualifying fill after a 365-day washout. The protocol allows refill gaps up to 30 days after supply runs out. A patient deviates if they switch to the other class, fail to refill within the gap rule, disenroll from observable pharmacy coverage, or receive a prohibited add-on defined in the protocol. Follow-up is artificially censored at the first deviation unless the outcome has already occurred. Fit a pooled logistic censoring model in person-months using baseline and time-updated covariates, form stabilized IPC weights, truncate extreme weights as pre-specified, then fit a weighted discrete-time outcome model and standardize 2-year cumulative incidence by assigned arm. The result is the effect that would have been expected if everyone in the eligible new-user population had remained adherent to their initial class under the observed covariate history assumptions.

Worked example

Scenario

A claims analyst compares new SGLT2i and DPP-4i initiators but wants the sustained-use effect rather than the usual-care initiation effect. The protocol says patients must remain on their index class, may have no refill gap beyond 30 days after supply runs out, and may not switch to the comparator class. The analyst censors person-time at the first protocol deviation and uses IPCW to adjust for the fact that deviation is prognostic.

Dataset

Simplified person-period records for a per-protocol analysis. The deviation month is censored unless the outcome occurred first.

Steps

• Lock each patient's assigned arm at the index fill, as in the ITT analysis.

• Build monthly exposure status from fills and days_supply. A gap longer than 30 days or a switch to the comparator class is a protocol deviation.

• P1001 switches at month 8 and has not yet had the outcome, so their SGLT2i person-time is artificially censored at month 8.

• P1002 has the outcome at month 5 before any deviation, so the event stays in the SGLT2i arm and is not censored.

• P2001 has a refill gap by month 4, so their DPP-4i follow-up is censored at month 4; similar still-adherent patients are up-weighted by IPCW to represent censored patients like P2001.

• Fit the censoring model using covariates measured before each month, multiply stabilized weights over time, then fit the weighted outcome model on uncensored person-months.

Result

The per-protocol dataset is a weighted person-period file. Each person starts in the same index arm as the ITT analysis, but follow-up stops at deviation and the remaining adherent person-time is reweighted to represent the original eligible target population under the full-adherence protocol.

Runnable example

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
import statsmodels.api as sm

def prepare_pp(pp):
    pp = pp.sort_values(["person_id", "month"]).copy()
    # Outcome month is retained even if deviation is also flagged; the event happened before censoring can remove it.
    pp.loc[pp["event"] == 1, "deviated"] = 0
    pp["uncensored"] = 1 - pp["deviated"]
    pp["t"] = pp["month"].astype(float)
    return pp

def stabilized_ipcw(pp, covariates):
    pp = prepare_pp(pp)
    rhs_den = "assigned_arm + t + I(t ** 2) + " + " + ".join(covariates)
    rhs_num = "assigned_arm + t + I(t ** 2)"
    den = smf.logit("uncensored ~ " + rhs_den, data=pp).fit(disp=False)
    num = smf.logit("uncensored ~ " + rhs_num, data=pp).fit(disp=False)
    pp["p_den"] = den.predict(pp).clip(0.01, 0.99)
    pp["p_num"] = num.predict(pp).clip(0.01, 0.99)
    pp["sw"] = (pp.groupby("person_id")["p_num"].cumprod() /
                pp.groupby("person_id")["p_den"].cumprod())
    pp["sw"] = pp["sw"].clip(upper=pp["sw"].quantile(0.99))
    return pp

def fit_per_protocol(pp, covariates):
    w = stabilized_ipcw(pp, covariates)
    analytic = w[w["deviated"] == 0].copy()
    model = smf.glm("event ~ assigned_arm + t + I(t ** 2)",
                    data=analytic,
                    family=sm.families.Binomial(),
                    freq_weights=analytic["sw"]).fit()
    risks = {}
    grid = analytic[["t"]].drop_duplicates().sort_values("t")
    for arm in sorted(analytic["assigned_arm"].unique()):
        pred_grid = grid.assign(assigned_arm=arm)
        hazards = model.predict(pred_grid).to_numpy()
        risks[arm] = float(1 - np.prod(1 - hazards))
    arms = sorted(risks)
    return {"model": model, "risk_by_arm": risks,
            "risk_difference": risks[arms[1]] - risks[arms[0]],
            "weights": w}

library(data.table)

prepare_pp <- function(pp) {
  setDT(pp); setorder(pp, person_id, month)
  pp[event == 1L, deviated := 0L]
  pp[, uncensored := 1L - deviated]
  pp[, t := as.numeric(month)]
  pp[]
}

stabilized_ipcw <- function(pp, covariates) {
  pp <- prepare_pp(copy(pp))
  fden <- as.formula(paste("uncensored ~ assigned_arm + t + I(t^2) +",
                           paste(covariates, collapse = " + ")))
  fnum <- uncensored ~ assigned_arm + t + I(t^2)
  den <- glm(fden, data = pp, family = binomial())
  num <- glm(fnum, data = pp, family = binomial())
  pp[, p_den := pmin(pmax(predict(den, pp, type = "response"), 0.01), 0.99)]
  pp[, p_num := pmin(pmax(predict(num, pp, type = "response"), 0.01), 0.99)]
  pp[, sw := cumprod(p_num) / cumprod(p_den), by = person_id]
  pp[, sw := pmin(sw, quantile(sw, 0.99))]
  pp[]
}

fit_per_protocol <- function(pp, covariates) {
  w <- stabilized_ipcw(pp, covariates)
  analytic <- w[deviated == 0L]
  fit <- glm(event ~ assigned_arm + t + I(t^2), data = analytic,
             family = binomial(), weights = sw)
  grid <- unique(analytic[, .(t)])[order(t)]
  arms <- sort(unique(analytic$assigned_arm))
  risk <- sapply(arms, function(a) {
    h <- predict(fit, cbind(grid, assigned_arm = a), type = "response")
    1 - prod(1 - h)
  })
  list(fit = fit, risk_by_arm = risk,
       risk_difference = unname(risk[2] - risk[1]), weights = w)
}