Formulary Restriction and Step-Therapy Natural Experiments

Formulary restriction and step-therapy natural experiments

use payer or benefit-design changes as externally timed shocks. A plan adds prior authorization, moves a drug from preferred to nonpreferred tier, imposes a step edit, removes a product from formulary, changes copayments, or caps reimbursed prescriptions. Because the rule is assigned at the plan, state, employer, PBM, or formulary-cell level, the primary estimand is usually the effect of the policy on the eligible population, not the effect of taking a drug among patients who successfully navigate the rule.

Core conceptual distinction

The intervention is the access rule. It may change initiation, switching, abandonment, adherence, dose delays, monitoring, and clinical outcomes through multiple pathways. That makes it a natural experiment for policy evaluation and a possible instrument for treatment receipt, but those are different analyses. A difference-in-differences or interrupted time-series design estimates the policy effect. An instrumental-variable analysis estimates a complier treatment effect only if the formulary shock affects the outcome exclusively through treatment received, which is often false because the shock also changes cost-sharing, delays, monitoring, and patient burden.

Pros, cons, and trade-offs

• Difference-in-differences vs interrupted time series. DiD is preferred when comparable untreated or not-yet-treated plans exist and parallel trends are plausible. ITS is useful for a single treated population with many pre/post periods, but it is vulnerable to secular market trends and concurrent guideline, rebate, or safety events.
• Policy effect vs treatment effect. The policy effect is directly relevant to payers and HTA: what happened after the rule? The treatment effect requires stronger assumptions because the policy can change more than treatment choice.
• Plan-level panel vs patient-level cohort. Plan-month panels make the policy timing and denominators transparent. Patient-level cohorts are needed for clinical outcomes and subgroups but require careful assignment of policy exposure, stable enrollment, and clustering at the policy unit.
• Formulary files vs observed claims. Formulary files establish intent and effective dates. Claims establish behavior. Rejected claims, prior-authorization denials, coupons, exceptions, and pharmacy benefit carve-outs are often missing, so observed fills can understate access friction.

When NOT to use - and when it is actively misleading

• Do not use a plan that adopted restriction because utilization or cost was already rising unless pre-trends are explicitly checked and bounded.
• Do not treat a formulary shock as an instrument if it also changes copayments, monitoring, paperwork burden, adherence, or time to treatment; those are direct pathways to outcomes.
• Do not compare fills before and after a rule without stable denominators. Enrollment churn, employer migration, Medicaid redetermination, or changing MA share can create a false policy effect.
• Do not ignore anticipation and grandfathering. Patients and prescribers may stockpile, accelerate starts, submit exceptions, or be exempt if already treated before the effective date.
• Do not use rejected or absent fills as a complete measure of access if the data do not capture rejected pharmacy claims, appeals, coupons, or out-of-plan cash purchases.

Data-source operational depth

Claims provide fills, switches, HCRU, and costs, but plan, benefit, formulary, rejected-claim, and denial data may be separate. Medicare FFS claims do not observe Medicare Advantage encounter completeness; Part D event data can capture fills but not all rejection and exception workflows. Commercial datasets may lose pharmacy or behavioral health carve-outs. EHRs can capture clinical outcomes and delays but usually miss the exact formulary rule and off-network fills. Registries capture disease severity and response but need claims or benefit files for policy assignment. Linked claims-formulary-EHR data are strongest, especially when effective dates, grandfathering, pharmacy rejections, clinical outcomes, and death are all observable.

Decision diagram

flowchart TD
  Policy["Formulary rule<br/>PA, step therapy, tier, cap"] --> Access["Access friction<br/>delay, abandonment, switching, copay"]
  Access --> Use["Drug utilization<br/>starts, adherence, discontinuation"]
  Use --> Outcomes["Clinical outcomes, HCRU, cost"]
  Policy --> Burden["Paperwork and monitoring burden"]
  Burden --> Outcomes
  Policy -. "IV only if no direct path except treatment" .-> Outcomes

Worked example

Scenario

A commercial plan adds step therapy for branded DPP-4 inhibitors on 2024-01-01. Similar plans administered by the same PBM do not change the rule. The analyst builds a plan-month panel from claims and formulary files.

Dataset

Illustrative plan-year rates of new DPP-4 initiations per 1,000 eligible members.

Steps

Result

The simple DiD contrast is (27 - 42) - (36 - 39) = -12 new DPP-4 starts per 1,000 members, interpreted as the policy effect if pre-trends and falsification checks are acceptable.

Trade-offs

Runnable example

import pandas as pd
import statsmodels.formula.api as smf

def fit_formulary_did(panel):
    panel = panel.copy()
    panel["treated_post"] = (
        (panel["first_policy_month"].notna()) &
        (panel["month"] >= panel["first_policy_month"])
    ).astype(int)
    panel["rate"] = 1000 * panel["new_starts"] / panel["eligible_member_months"]
    fit = smf.wls(
        "rate ~ treated_post + C(plan_id) + C(month)",
        data=panel,
        weights=panel["eligible_member_months"],
    ).fit(cov_type="cluster", cov_kwds={"groups": panel["plan_id"]})
    return fit

library(fixest)

fit_formulary_did <- function(panel) {
  panel$treated_post <- as.integer(!is.na(panel$first_policy_month) &
                                   panel$month >= panel$first_policy_month)
  panel$rate <- 1000 * panel$new_starts / panel$eligible_member_months
  feols(rate ~ treated_post | plan_id + month,
        data = panel,
        weights = ~ eligible_member_months,
        cluster = ~ plan_id)
}

data panel_rates;
  set work.panel;
  treated_post = (first_policy_month ne . and month >= first_policy_month);
  rate = 1000 * new_starts / eligible_member_months;
run;

proc surveyreg data=panel_rates;
  cluster plan_id;
  weight eligible_member_months;
  class plan_id month;
  model rate = treated_post plan_id month / solution;
  ods select ParameterEstimates;
run;

Citations

1. [1]Huskamp HA, Deverka PA, Epstein AM, Epstein RS, McGuigan KA, Frank RG. The effect of incentive-based formularies on prescription-drug utilization and spending. New England Journal of Medicine. 2003;349(23):2224-2232.

1. [2]Soumerai SB, Ross-Degnan D, Avorn J, McLaughlin TJ, Choodnovskiy I. Effects of Medicaid drug-payment limits on admission to hospitals and nursing homes. New England Journal of Medicine. 1991;325(15):1072-1077.
2. [3]Zhang Y, Adams AS, Ross-Degnan D, Zhang F, Soumerai SB. Effects of prior authorization on medication discontinuation among Medicaid beneficiaries with bipolar disorder. Psychiatric Services. 2009;60(4):520-527.

1. [4]Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics. 2002;27(4):299-309.
2. [5]Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Annual Review of Public Health. 2018;39:453-469.