Synthetic Control Method

In plain language

The synthetic control method estimates what would have happened to a single region or health system if an intervention had never occurred, by building a stand-in from a weighted blend of similar untreated places. You choose a set of comparison regions (the donor pool), assign each a weight so that the blend closely tracks the treated region's outcome history before the intervention, then read off the post-intervention gap between what was observed and what the blend predicts. The method works for aggregate questions such as 'did this state policy reduce drug initiation rates?' and it makes the counterfactual auditable because the donor weights are visible numbers you can inspect and challenge.

The synthetic control method (SCM) estimates the effect of an intervention that affects one (or a few) aggregate unit(s) — a state that adopts a policy, a health system that rolls out a program, a country that changes drug regulation — for which no single untreated unit is a credible comparison. Instead of picking one comparator, SCM constructs a synthetic control: a convex, weighted combination of units in a donor pool of untreated units, with non-negative weights summing to one, chosen so that the weighted donor outcomes track the treated unit's outcome and its predictors as closely as possible during the pre-intervention period. The weights W minimize the pre-period discrepancy ||X1 − X0·W|| between the treated unit's predictor/outcome vector X1 and the donors' matrix X0 (typically a nested optimization that also weights which predictors matter, V). The treatment effect at each post-period t is the gap α_t = Y1_t − Σ_j w_j·Y0_jt between the treated unit's observed outcome and its synthetic counterpart. The method's appeal is twofold: the counterfactual is transparent (you can read off which donors get weight and verify the pre-period fit), and the convexity constraint guards against extrapolation beyond the donors' support. SCM formalizes and disciplines the intuition behind a "comparison region" that practitioners had long chosen by hand.

Core conceptual distinction

SCM is the single-treated-unit generalization of difference-in-differences. (1) Relation to DiD: DiD uses an equal-weighted (or regression-weighted) comparison group and assumes parallel trends; SCM instead data-drives the donor weights to match the treated unit's pre-period path, relaxing the assumption that any one comparator is parallel and replacing it with the requirement of good pre-period fit and a stable post-period relationship. When the pre-fit is good and the donor pool is rich, SCM nests DiD as a special case. (2) Donor pool and pre-period fit are the whole game: the donor pool must contain only untreated units plausibly driven by the same factors as the treated unit and not contaminated by the intervention or by their own idiosyncratic shocks; the pre-period fit (a small pre-period root-mean-square prediction error, RMSPE) is the prerequisite for trusting the post-period gap — a poor pre-fit means the synthetic control is not a credible counterfactual and the gap is uninterpretable. (3) Inference is by placebo/permutation, not classical SEs: with one treated unit there is no sampling distribution in the usual sense, so inference proceeds by placebo (in-space) tests — re-estimating SCM pretending each donor was treated and asking whether the true treated unit's post/pre RMSPE ratio is extreme relative to the placebo distribution — and in-time placebos (a fake intervention date in the pre-period that should show no gap). This permutation logic, not a t-statistic, is what produces a p-value.

Pros, cons, and trade-offs

- vs difference-in-differences (`difference-in-differences-staggered-adoption-rwe`): SCM data-drives the comparison weights to match the treated unit's pre-trajectory, so it does not require any single donor to be parallel and makes the counterfactual auditable; DiD is simpler, supports many treated units and standard inference, and is more efficient when a credible parallel comparison group exists. Prefer SCM for one (or few) treated aggregate units with a long pre-period and a rich donor pool; prefer DiD with many treated units, short pre-periods, or when classical inference and covariate adjustment are needed. SCM nests DiD when the optimal weights are uniform. - vs interrupted time series (`interrupted-time-series-rwe`): ITS uses only the treated unit's own series and extrapolates its pre-trend as the counterfactual; SCM borrows strength from untreated donors to construct the counterfactual, which protects against confounding by a co-timed shock that also hits the treated series (if the shock hits donors too, the synthetic control absorbs it). Prefer SCM when good donor units exist and co-timed shocks are a worry; prefer ITS when no comparable donor units are available and the pre-trend is stable. - vs instrumental variables / target-trial (`instrumental-variables-pharmacoepi-rwe`, `target-trial-emulation`): Those are individual-level designs for many units with measured (or instrumented) confounding; SCM is an aggregate, small-N design whose identification rests on pre-period fit and donor-pool validity rather than measured confounders or an instrument. Prefer SCM when the unit of intervention is itself aggregate (a state/system/country) and the effective sample of treated units is one; prefer individual-level designs when many units are exposed and patient-level data and confounders exist.

When to use

A single (or very few) treated aggregate unit(s); a long, well-measured pre-intervention outcome series (a common rule of thumb is many pre-periods, ideally a decade-plus of annual data or many quarters); a donor pool of untreated units that are comparable, uncontaminated by the intervention, and free of their own large idiosyncratic shocks; an intervention whose effect on an aggregate metric (a state's hospitalization rate, a health system's per-member cost, a region's drug uptake) is the question. SCM is the standard tool for policy/program evaluations where the treated unit is a geography or organization.

When NOT to use — and when it is actively misleading or dangerous

- Poor pre-intervention fit. If the synthetic control cannot reproduce the treated unit's pre-period trajectory (large pre-period RMSPE), it is not a valid counterfactual and the post-period gap is meaningless. Reporting an effect on top of a bad pre-fit is the single most dangerous SCM misuse — always show the pre-period fit and the RMSPE. - Contaminated or shocked donor pool. Donors affected by the same or a similar intervention (spillover), or experiencing their own large idiosyncratic shocks during the study window, corrupt the synthetic control. Curate the donor pool and run leave-one-out checks dropping high-weight donors. - Extrapolation / interpolation bias. If the treated unit's predictors lie outside the convex hull of the donors, no convex weighting can match it and the method silently extrapolates; if donors are very dissimilar, interpolation across them is unreliable. Check that the treated unit is inside the donors' support. - Few pre-periods or volatile outcome. A short or noisy pre-period cannot pin the weights, so over-fitting to noise inflates the apparent post-period gap; the in-time placebo will expose this. - Over-reading a single gap as significant. Without placebo/permutation inference, a visually large gap means little — aggregate series are volatile, and many placebo units will show gaps as large by chance. Report the placebo distribution and the post/pre RMSPE-ratio p-value, not just the gap.

Data-source operational depth

- Claims (FFS vs MA): The treated and donor units are usually geographies (states, regions, plans) and the outcome is an aggregated rate or cost (per-1,000 hospitalizations, PMPM cost) over a stable denominator. A standing failure mode: shifting Medicare Advantage penetration across states and over time changes which population is FFS-observable, so a region's measured FFS rate can move for compositional reasons unrelated to the intervention — hold the denominator definition constant, restrict to a consistently observable population, and confirm donor regions did not experience their own coverage-mix shocks during the window. Use predictors (age/sex mix, baseline comorbidity, baseline outcome levels) that are stably measured across all units. - EHR / health-system units: When treated and donor units are provider organizations or systems, documentation and coding-policy differences across systems create level differences the pre-period fit must reconcile; ensure outcome and predictor definitions are harmonized across all donor systems before fitting, and treat a system-specific EHR transition as a potential idiosyncratic shock that disqualifies a donor. - Registry / linked: Population registries (cancer, perinatal, disease registries) at the geographic level give clean aggregate outcomes for SCM; verify reporting completeness is comparable across donor regions and calendar periods so a reporting-lag difference is not mistaken for a treatment gap.

Worked claims example

Question: did a 2018 state-level prior-authorization policy for high-cost specialty drug class X reduce the state's age-standardized initiation rate per 100,000 adults, using a commercial + Medicare FFS multi-state claims database? (1) Units and outcome: treated unit = the policy state; donor pool = 24 states with no comparable policy during 2010-2022 and no large coverage-mix shock; outcome = annual age/sex-standardized initiation rate over a constant FFS-observable denominator. (2) Predictors: pre-2018 initiation levels at several lags, age/sex distribution, baseline comorbidity index, baseline specialty-drug spend. (3) Fit: nested optimization selects donor weights (e.g., 0.34 State B, 0.22 State F, 0.19 State K, 0.15 State P, 0.10 State T; all others 0) reproducing the treated state's 2010-2017 trajectory with a small pre-period RMSPE (close tracking, visually overlapping lines). (4) Effect: post-2018 the observed initiation rate falls below the synthetic control by a widening gap, averaging −6.4 initiations per 100,000/year over 2018-2022. (5) Inference: in-space placebos re-estimate SCM treating each of the 24 donors as "treated"; the policy state's post/pre RMSPE ratio (≈ 5.1) is the largest of 25, giving a permutation p ≈ 1/25 = 0.04; an in-time placebo (fake 2015 intervention) shows no pre-2018 gap, and leave-one-out re-estimation dropping each high-weight donor leaves the effect stable. The interpretation is reported as an effect for this state under this policy, with the donor weights, pre-fit, and placebo distribution shown in full.

Interpreting the output

Using the worked example: the synthetic control's pre-period 2017 rate (40.99 per 100,000) tracks the policy state's observed rate (41.0) to within 0.01. In 2019, the synthetic control predicts 42.99 per 100,000 while the policy state observed 36.59, yielding a gap of 36.59 − 42.99 = −6.40 per 100,000.

Formal interpretation: The estimated effect is −6.40 new initiations per 100,000 adults in 2019 for this specific treated state. The synthetic control does not produce an ATE or ATT in the conventional sense — it estimates the effect for a single unit (the policy state), comparing its observed post-period outcome to the counterfactual trajectory constructed from the donor-weighted average. The credibility of the −6.40 gap rests entirely on pre-period fit quality: a synthetic control that closely tracked the policy state before the intervention (pre-period discrepancy of 0.01 per 100,000) provides a plausible stand-in for the unobserved counterfactual. Inference is permutation-based: the observed gap is significant if it exceeds most donor-state placebo gaps. Results apply only to this treated state and this policy — they do not average across many states or generalize to different policy contexts.

Practical interpretation: The prior-authorization policy is estimated to have reduced new drug initiations by roughly 6.4 per 100,000 adults per year compared with what a weighted combination of similar donor states experienced over the same period. The quality of this claim depends on how well the donor pool tracked the policy state before the policy — excellent in this example (discrepancy of 0.01). A permutation p-value from placebo tests across donor states quantifies how unusual the observed gap is.

Worked example

Scenario

A state passes a prior-authorization policy for a specialty drug class in 2018. We want to know whether the policy reduced the state's initiation rate (new users per 100,000 adults per year). We have annual initiation rates for the policy state and five untreated donor states from 2010 through 2022. The synthetic control is built by assigning each donor a weight so that the weighted average of donor rates tracks the policy state's rate as closely as possible in 2010-2017, then the 2019 gap between the policy state and its synthetic stand-in estimates the effect.

Dataset

Annual initiation rate (new users per 100,000 adults) for the policy state and five donor states. Weights are chosen by the SCM optimizer to minimize pre-period discrepancy.

Steps

• Compute the synthetic control's 2017 (pre-period) rate as the weighted average of donors: (0.34 x 42.0) + (0.22 x 38.0) + (0.19 x 45.0) + (0.15 x 36.0) + (0.10 x 44.0) = 14.28 + 8.36 + 8.55 + 5.40 + 4.40 = 40.99 per 100,000.

• Compare to the policy state's observed 2017 rate of 41.0 per 100,000 — the synthetic control is only 0.01 off, confirming excellent pre-period fit. This close tracking in the years before 2018 is what makes the post-period comparison credible.

• Compute the synthetic control's 2019 (post-period) rate: (0.34 x 44.0) + (0.22 x 40.0) + (0.19 x 47.0) + (0.15 x 38.0) + (0.10 x 46.0) = 14.96 + 8.80 + 8.93 + 5.70 + 4.60 = 42.99 per 100,000.

• The policy state's observed 2019 rate is 36.59 per 100,000. The gap is 36.59 minus 42.99 = -6.40 per 100,000 — the state initiated 6.40 fewer patients per 100,000 adults than its synthetic stand-in predicted without the policy.

• Verify the weights sum to 1: 0.34 + 0.22 + 0.19 + 0.15 + 0.10 = 1.00. All weights are between 0 and 1. The synthetic control is a valid weighted average, not an extrapolation.

Result

Estimated effect: -6.40 initiations per 100,000 adults in 2019, matching the file's reported average effect of -6.4/year over 2018-2022. The negative gap means the policy state's observed rate fell 6.40 below what the donor-weighted synthetic control predicted would have happened without the policy. Placebo tests across the 5 donor states confirmed this gap is larger than any donor's spurious gap, giving a permutation p-value of approximately 1/6 = 0.17 (with 5 donors) — highlighting why a richer donor pool matters for inference.

Runnable example

import numpy as np
import pandas as pd
from pysyncon import Dataprep, Synth
from pysyncon.inference import SpacePlacebo

# Long panel: one row per (unit, year). 'state' is the unit; 'init_rate' the outcome.
panel = pd.read_csv("state_initiation_panel.csv")   # columns: state, year, init_rate, age_mix, comorbidity, spend

prep = Dataprep(
    foo=panel,
    predictors=["age_mix", "comorbidity", "spend"],
    predictors_op="mean",
    dependent="init_rate",
    unit_variable="state",
    time_variable="year",
    treatment_identifier="PolicyState",
    controls_identifier=[s for s in panel["state"].unique() if s != "PolicyState"],
    time_predictors_prior=range(2010, 2018),        # pre-intervention period (fit window)
    time_optimize_ssr=range(2010, 2018),
)
synth = Synth()
synth.fit(dataprep=prep)                            # solve donor weights W (convex, sum to 1)
print("Donor weights:", synth.weights().round(2))   # auditable contribution of each donor

# Treated-minus-synthetic gap = estimated effect per post-period year.
gaps = synth.gaps(time_period=range(2010, 2023))
print(gaps.loc[2018:2022])                          # post-intervention effect path

# Placebo / permutation inference: rank the treated unit's post/pre RMSPE ratio vs donors.
placebo = SpacePlacebo(dataprep=prep, synth=Synth())
placebo.fit()
print("Permutation p-value:", placebo.summary())

library(Synth)

# `panel` is a long data.frame: state, year, init_rate, age_mix, comorbidity, spend.
dp <- dataprep(
  foo = panel,
  predictors = c("age_mix", "comorbidity", "spend"),
  predictors.op = "mean",
  dependent = "init_rate",
  unit.variable = "state_id",
  time.variable = "year",
  treatment.identifier = policy_state_id,
  controls.identifier  = donor_state_ids,
  time.predictors.prior = 2010:2017,        # pre-intervention fit window
  time.optimize.ssr     = 2010:2017,
  time.plot             = 2010:2022
)

so <- synth(dp)                              # solve convex donor weights W and predictor weights V
print(round(so$solution.w, 2))               # donor weights (auditable counterfactual)

# Pre-period fit (RMSPE) gates interpretation; gaps.plot shows observed - synthetic.
tab <- synth.tab(synth.res = so, dataprep.res = dp)
print(tab$tab.pred)                          # treated vs synthetic predictor balance
gaps.plot(synth.res = so, dataprep.res = dp, Main = "Observed minus synthetic (effect path)")
path.plot(synth.res = so, dataprep.res = dp, Main = "Treated vs synthetic control")