Formulary Restriction and Step-Therapy Natural Experiments

In plain language

A formulary natural experiment uses a real access rule change, such as a prior authorization or step-therapy requirement, as the event to study. The key question is what changed after the rule compared with what would have happened without it. The answer is usually a policy effect on access, switching, adherence, outcomes, or cost, not a simple drug effect.

Formulary restriction and step-therapy natural experiments

use payer or benefit-design changes as externally timed shocks. A plan adds prior authorization, moves a drug from preferred to nonpreferred tier, imposes a step edit, removes a product from formulary, changes copayments, or caps reimbursed prescriptions. Because the rule is assigned at the plan, state, employer, PBM, or formulary-cell level, the primary estimand is usually the effect of the policy on the eligible population, not the effect of taking a drug among patients who successfully navigate the rule.

Core conceptual distinction

The intervention is the access rule. It may change initiation, switching, abandonment, adherence, dose delays, monitoring, and clinical outcomes through multiple pathways. That makes it a natural experiment for policy evaluation and a possible instrument for treatment receipt, but those are different analyses. A difference-in-differences or interrupted time-series design estimates the policy effect. An instrumental-variable analysis estimates a complier treatment effect only if the formulary shock affects the outcome exclusively through treatment received, which is often false because the shock also changes cost-sharing, delays, monitoring, and patient burden.

Pros, cons, and trade-offs

- Difference-in-differences vs interrupted time series. DiD is preferred when comparable untreated or not-yet-treated plans exist and parallel trends are plausible. ITS is useful for a single treated population with many pre/post periods, but it is vulnerable to secular market trends and concurrent guideline, rebate, or safety events. - Policy effect vs treatment effect. The policy effect is directly relevant to payers and HTA: what happened after the rule? The treatment effect requires stronger assumptions because the policy can change more than treatment choice. - Plan-level panel vs patient-level cohort. Plan-month panels make the policy timing and denominators transparent. Patient-level cohorts are needed for clinical outcomes and subgroups but require careful assignment of policy exposure, stable enrollment, and clustering at the policy unit. - Formulary files vs observed claims. Formulary files establish intent and effective dates. Claims establish behavior. Rejected claims, prior-authorization denials, coupons, exceptions, and pharmacy benefit carve-outs are often missing, so observed fills can understate access friction.

When to use

Use this design when a restriction has a known effective date, is assigned above the patient level, and produces observable pre/post data. Strong use cases include step therapy for diabetes or specialty drugs, prior authorization for antipsychotics or biologics, preferred-tier changes, nonmedical switching, Medicaid drug caps, Medicare Part D formulary changes, and PBM or employer benefit redesign. Outcomes can include new starts, discontinuation, switching to required first-line therapy, time to therapy, abandonment, adherence, adverse events, disease control, HCRU, and total cost.

When NOT to use - and when it is actively misleading

- Do not use a plan that adopted restriction because utilization or cost was already rising unless pre-trends are explicitly checked and bounded. - Do not treat a formulary shock as an instrument if it also changes copayments, monitoring, paperwork burden, adherence, or time to treatment; those are direct pathways to outcomes. - Do not compare fills before and after a rule without stable denominators. Enrollment churn, employer migration, Medicaid redetermination, or changing MA share can create a false policy effect. - Do not ignore anticipation and grandfathering. Patients and prescribers may stockpile, accelerate starts, submit exceptions, or be exempt if already treated before the effective date. - Do not use rejected or absent fills as a complete measure of access if the data do not capture rejected pharmacy claims, appeals, coupons, or out-of-plan cash purchases.

Data-source operational depth

Claims provide fills, switches, HCRU, and costs, but plan, benefit, formulary, rejected-claim, and denial data may be separate. Medicare FFS claims do not observe Medicare Advantage encounter completeness; Part D event data can capture fills but not all rejection and exception workflows. Commercial datasets may lose pharmacy or behavioral health carve-outs. EHRs can capture clinical outcomes and delays but usually miss the exact formulary rule and off-network fills. Registries capture disease severity and response but need claims or benefit files for policy assignment. Linked claims-formulary-EHR data are strongest, especially when effective dates, grandfathering, pharmacy rejections, clinical outcomes, and death are all observable.

Worked example

Scenario

A commercial plan adds step therapy for branded DPP-4 inhibitors on 2024-01-01. Similar plans administered by the same PBM do not change the rule. The analyst builds a plan-month panel from claims and formulary files.

Dataset

Illustrative plan-year rates of new DPP-4 initiations per 1,000 eligible members.

Steps

• Verify the rule effective date and whether existing users were grandfathered.

• Define eligible members with stable medical and pharmacy enrollment and type 2 diabetes.

• Count new DPP-4 starts from NDC fills after a 365-day DPP-4 washout.

• Fit a DiD model with plan and month fixed effects, clustering at plan, and inspect event-study leads for pre-trend divergence and anticipation.

• Run falsification outcomes such as statin initiation and a placebo policy date.

Result

The simple DiD contrast is (27 - 42) - (36 - 39) = -12 new DPP-4 starts per 1,000 members, interpreted as the policy effect if pre-trends and falsification checks are acceptable.

Runnable example

import pandas as pd
import statsmodels.formula.api as smf

def fit_formulary_did(panel):
    panel = panel.copy()
    panel["treated_post"] = (
        (panel["first_policy_month"].notna()) &
        (panel["month"] >= panel["first_policy_month"])
    ).astype(int)
    panel["rate"] = 1000 * panel["new_starts"] / panel["eligible_member_months"]
    fit = smf.wls(
        "rate ~ treated_post + C(plan_id) + C(month)",
        data=panel,
        weights=panel["eligible_member_months"],
    ).fit(cov_type="cluster", cov_kwds={"groups": panel["plan_id"]})
    return fit

library(fixest)

fit_formulary_did <- function(panel) {
  panel$treated_post <- as.integer(!is.na(panel$first_policy_month) &
                                   panel$month >= panel$first_policy_month)
  panel$rate <- 1000 * panel$new_starts / panel$eligible_member_months
  feols(rate ~ treated_post | plan_id + month,
        data = panel,
        weights = ~ eligible_member_months,
        cluster = ~ plan_id)
}