Overlap Weights and Modern Propensity Weighting

In plain language

When comparing two treatments in real-world data, methods that reweight patients to make the two groups look similar can accidentally assign enormous statistical weight to patients who look nothing like the other group, letting a few unusual patients dominate the results. Overlap weights fix this by giving the most weight to patients who could plausibly have received either treatment and down-weighting everyone else, keeping all weights naturally bounded between zero and one without any arbitrary trimming decision. The trade-off is that the result applies to the subgroup of patients where a clinician's choice genuinely could have gone either way — the clinical equipoise population — not to every patient who happened to receive one drug. When the research question is about patients who actually used a drug (for a safety signal, for example), a different weighting approach that targets the treated population is needed instead.

Why inverse-probability weighting explodes at the propensity score tails

Standard inverse-probability-of-treatment weighting (IPTW) assigns weight 1/e(X) to each treated patient and 1/(1 minus e(X)) to each control patient, where e(X) is the estimated propensity score. When a patient has a propensity score close to one — their baseline covariates strongly predict that they receive the treatment — the control-arm weight 1/(1 minus e(X)) becomes enormous. A control patient with e(X) = 0.9 receives a weight of 1/(1 minus 0.9) = 10, while a treated patient with the same propensity score receives only 1/0.9 ≈ 1.11 — a nine-fold disparity arising purely from which arm the patient happened to receive. This is not a rounding artifact; it is structural. In real high-dimensional claims studies, propensity scores near zero and one are common when the two drugs serve meaningfully different patient populations, and a handful of extreme patients can dominate the entire pseudo-population. Analysts have traditionally responded with ad hoc weight truncation (clipping at the 1st/99th or 5th/95th percentile), but such thresholds are arbitrary, create specification bias if chosen after seeing results, and must be pre-specified to be credible in a regulatory or HTA submission. Even after truncation, variance inflation from near-extreme weights can be substantial, and the trimmed subjects represent a silent change in the estimand population.

Overlap weights: definition and the bounded-weight property

Li, Morgan and Zaslavsky (2018) proposed assigning each patient a weight equal to the probability that the patient would have been assigned to the opposite arm: treated patients receive weight w = 1 minus e(X) and control patients receive weight w = e(X). These are called overlap weights because they concentrate statistical mass in the region of propensity score overlap — the patients whose scores are intermediate, indicating genuine clinical uncertainty about which treatment they would receive. Two properties of overlap weights are mathematically guaranteed regardless of sample size. First, all weights are strictly bounded within the open unit interval (0, 1); no patient can receive a weight exceeding one, and no trimming decision is ever needed. Second, among all balancing weight functions that achieve exact mean balance on every covariate entering the propensity score model, overlap weights have the smallest weighted-estimator variance — a global optimality property proved by Li et al. Concretely, the overlap weight function is proportional to e(X) times (1 minus e(X)), which peaks at e(X) = 0.5 (perfect equipoise) and tapers smoothly to zero at both extremes. The exact mean balance holds as an algebraic identity for any covariates whose means appear in the PS model: the weighted mean of each such covariate is identical in the treated and control arms after overlap weighting, without iterative optimization. This algebraic balance guarantee differentiates overlap weights from IPTW, where balance is an asymptotic property that may fail badly in finite samples with near-positivity violations.

The ATO estimand: clinical equipoise, not the full eligible population

Overlap weights do not estimate the average treatment effect in the full eligible population (ATE) or in the treated subpopulation (ATT). They target the average treatment effect in the overlap population (ATO) — the smoothly-weighted average effect in patients for whom clinical choice could plausibly go either way, indexed by w = e(X) times (1 minus e(X)). This population is not a fixed clinical subgroup; it is a smooth reweighting defined by the propensity score model, which means its composition changes if the PS model changes. The ATO is the most policy-relevant estimand for comparative effectiveness research in therapeutic areas where two drugs are genuinely co-prescribed for the same indication and clinicians choose based on patient-level factors the analyst has measured: two antidiabetic drugs used interchangeably across the glycemic spectrum; two biologics in inflammatory bowel disease with largely overlapping eligibility; two anticoagulants with similar label populations. The ATO is NOT appropriate in several settings: regulatory submissions requesting the effect in patients who actually received the drug (ATT) — for a safety signal, a post-approval commitment study, or a label-expansion dossier — require ATT weighting or 1:1 matching, not ATO. When the target population is anchored to a specific clinical eligibility rule (e.g., "all patients who initiated Drug A"), the ATO population may not map to that clinical group. Any label claim or clinical-practice guideline update must describe the target population in recognizable clinical terms; the ATO's smooth propensity-score-indexed population often cannot be translated into a patient group that prescribers can identify prospectively.

The weighting-target zoo: ATE, ATT, ATO, matching weights, and entropy weights

Five major weight families target distinct estimands and populations, and the choice among them is a substantive decision that must be pre-specified. IPTW for the ATE weights treated patients by Pr(A=1)/e(X) and controls by Pr(A=0)/(1 minus e(X)), targeting the whole eligible population; it is the most sensitive to positivity violations and produces the widest weight range. Standardized mortality ratio (SMR) weighting for the ATT weights treated patients at 1 and controls at e(X)/(1 minus e(X)), targeting the treated population and naturally suited to post-marketing safety signals about actual drug users. Overlap weights for the ATO target the clinical equipoise population with bounded weights and exact covariate balance. Matching weights (Li and Greene 2013) assign each patient the minimum of e(X) and 1 minus e(X), yielding the largest subpopulation that can be exactly balanced while remaining matchable — a population anchored to matched pairs. Entropy balancing selects weights by directly satisfying balance constraints rather than by a weight formula derived from the PS. None of these estimands is cosmetically interchangeable: when the treatment effect is heterogeneous across the PS distribution, ATE, ATT, and ATO can differ substantially in magnitude and can even differ in sign. Pre-specifying the estimand before examining outcomes is not optional — it is what distinguishes a confirmatory analysis from post hoc effect shopping.

Entropy balancing

Entropy balancing (Hainmueller 2012) searches for the set of weights that (a) directly satisfies user-specified balance constraints — usually exact first-moment balance on all listed covariates — while (b) staying as close as possible to uniform weights in a Kullback-Leibler information sense. The weights are found by solving a convex optimization problem rather than by a closed-form formula. Unlike overlap weights, entropy balancing bypasses the propensity score model entirely: there is no e(X) to estimate; the optimizer works directly on covariate moments. The analyst specifies which covariates must be balanced (and to which moments — first, second, cross-product) and the procedure finds the minimum-entropy-deviation weights that satisfy those constraints. The estimand of entropy balancing is ATT-style when only the control group is reweighted to match the treated covariate distribution, or ATE-style when both groups are reweighted toward a target distribution. Entropy balancing is more transparent about what balance means in a given analysis — the analyst explicitly lists the constraints — but does not produce a single named causal estimand as cleanly as overlap weights do. For regulatory submissions where the estimand must be pre-specified and interpretable, overlap weights are generally more defensible.

Covariate Balancing Propensity Score (CBPS)

The Covariate Balancing Propensity Score (Imai and Ratkovic 2014) estimates the propensity score model under the dual constraint that the model fits the treatment assignment (as in standard logistic regression) AND the resulting weights simultaneously achieve covariate balance. CBPS is a generalized method of moments estimator that enforces both the score equation and the balance equation as moment conditions, producing a PS model that is more robust to logistic misspecification than a standard logistic regression PS. For practical RWE use, CBPS is a drop-in replacement for logistic regression in any PS workflow — the estimated e(X) can feed into any downstream weight formula (ATE, ATT, ATO). Both CBPS and entropy balancing are implemented in the WeightIt R package via method = "cbps" and method = "ebal". For most large-N claims datasets where logistic regression is stable, the practical difference between a well-specified logistic PS and CBPS is small; CBPS adds value when the PS model includes flexible terms such as splines or interaction sets that are sensitive to specification.

Diagnostics: unchanged from standard IPTW

The post-weighting diagnostic checklist for overlap weights is identical to that for IPTW. Standardized mean differences (SMDs) in the weighted sample must fall below 0.1 for all covariates in the PS model, including squared terms and interactions if those entered the model. The effective sample size ESS = (sum of w)^2 / (sum of w^2) must be reported; overlap weights often produce a higher ESS than unstabilized IPTW because bounded weights have lower dispersion than extreme ones, but this is not guaranteed when IPTW weights are stabilized or when the PS distribution is concentrated near 0.5 for most patients. Any ESS improvement reflects the narrower ATO estimand, not better data quality. An overlap plot of the raw PS distribution in both arms — before weighting — confirms whether actual data overlap exists; the bounded-weight property does not create overlap that is absent in the raw data, it merely avoids assigning disproportionate influence to patients near the distribution extremes. The weight distribution should be plotted alongside the ESS to confirm that near-zero weights are only assigned to far-from-equipoise patients where down- weighting is scientifically appropriate.

Pros, cons, and trade-offs

Overlap weights versus IPTW: IPTW targets the ATE — the full eligible population — and has a transparent connection to the target trial protocol; it is the natural choice when the research question asks what would happen if everyone in the eligible population switched drugs. Its cost is susceptibility to extreme weights and variance inflation when overlap is poor, even after truncation. Overlap weights eliminate the truncation decision entirely, produce the minimum- variance balanced estimator, and achieve exact mean covariate balance in any sample; their cost is the narrower ATO estimand, which cannot be used whenever the policy question requires ATT or ATE and when the equipoise population cannot be described in actionable clinical terms.

Overlap weights versus entropy balancing: both achieve exact covariate mean balance, but overlap weights derive their population from an estimated PS and target a defined ATO estimand, while entropy balancing directly targets a user-specified covariate distribution with no PS estimation step. Entropy balancing adds no positivity penalty — the optimization may produce extreme weights if the specified constraints are nearly infeasible — while overlap weights are bounded by construction. For regulatory submissions, overlap weights' clear estimand formulation is preferred.

Overlap weights versus CBPS: CBPS is a PS estimation method, not a weight scheme; the two are composable — estimate e(X) via CBPS, then compute overlap or ATT or ATE weights from e(X). Using CBPS-estimated scores with overlap weights combines the misspecification robustness of CBPS with the bounded-weight guarantee of overlap weighting.

When to use

Use overlap weights when the PS distribution shows moderate overlap concerns and IPTW produces extreme weights whose truncation threshold would be disputed; when the policy question is comparative effectiveness among patients where clinicians genuinely choose between two drugs (not post-marketing safety among current users); when the analysis is not a regulatory submission requiring the ATT; and when exact covariate mean balance without a separate optimization step is methodologically desirable. Overlap weights are the modern default for head-to-head comparative effectiveness studies submitted to HTA bodies in therapeutic areas where both drugs are widely co-prescribed. Entropy balancing is preferred when the PS model is difficult to specify and the analyst wants to express balance constraints directly. CBPS is preferred when a PS model is needed downstream for doubly-robust estimation and logistic regression misspecification is a concern.

When NOT to use

Do not use overlap weights when the estimand must be the average treatment effect in the treated (ATT): any safety signal, post-approval commitment study, or label-support analysis targeting actual drug users requires ATT weighting or matched-cohort analysis, not ATO. Do not use overlap weights when the ATO population cannot be described in clinically recognizable terms for the guideline or regulatory dossier; if reviewers cannot identify who the clinical equipoise patients are, the estimand is not actionable. Do not use any weighting method as a substitute for design: a poorly defined time zero, a wrong comparator, or a prevalent-user cohort produces a biased ATO estimate just as it produces biased ATE or ATT estimates — the overlap weight diagnostics will look clean while the underlying cohort is structurally biased. Do not cite the bounded-weight property as evidence of adequate data overlap — weights are bounded by construction regardless of the raw PS distribution; inspect the raw PS histogram before choosing any weighting method. Do not apply overlap weighting to sparse-outcome studies with very low event counts; weighted survival analyses are unstable in small effective samples regardless of which weight formula is used.

Interpreting the output

Consider an overlap-weighted comparative effectiveness analysis of two oral antidiabetic agents producing a weighted risk difference of minus 0.032 (95% CI minus 0.052 to minus 0.012) for a composite cardiovascular endpoint, with all post-weighting SMDs below 0.08 and ESS of 6,420 from an original cohort of 8,100 patients.

Formal interpretation: The risk difference of minus 0.032 is the estimated average treatment effect in the overlap population (ATO) — the smoothly-weighted subpopulation indexed by w = e(X) times (1 minus e(X)). This is a marginal (population-averaged) effect in the ATO- weighted pseudo-population, not a conditional effect within covariate strata and not the ATE or ATT. The 95% confidence interval (minus 0.052 to minus 0.012), computed with robust sandwich standard errors that account for weight heterogeneity in the outcome model, means that under repeated sampling from the same data-generating process, 95% of such intervals would contain the true ATO risk difference. Sandwich SEs do not propagate propensity-score estimation uncertainty from the first stage; bootstrap or M-estimation is required if that uncertainty must be reflected in the interval. The estimate is a valid causal estimate only under three core assumptions: conditional exchangeability given all measured baseline covariates X, consistency (the treatment versions are well-defined), and positivity within the ATO target population (every patient has a nonzero probability of receiving either treatment). Robustness to PS model misspecification is better than with IPTW because bounded weights limit the influence of any single patient, but PS misspecification is not eliminated.

Practical interpretation: Among patients in the clinical-equipoise population (ATO), Drug A was associated with approximately 3.2 fewer events per 100 patients compared with Drug B. This is not the effect for all Drug A users, not the effect in the sickest patients essentially guaranteed one treatment, and not an estimate that directly generalizes to a trial that enrolled by strict eligibility criteria. The ESS of 6,420 is a weight-dispersion summary — the equivalent number of equal-weight observations yielding the same precision — indicating the analysis retains substantial statistical information; it is not a count of patients who receive meaningful weight, as all 8,100 patients receive nonzero overlap weights. Because the ATO population is defined by the PS model, it cannot be identified prospectively from clinical criteria alone; any label or guideline language derived from this estimate must acknowledge the estimand as the comparative effectiveness population identified by clinical equipoise in the observed prescription data, not a fixed patient subgroup.

Worked example

Scenario

An analyst compares Drug A versus Drug B in a commercial claims database. After fitting a logistic regression propensity score model, four representative patients illustrate how IPTW and overlap weights differ. Patients 101 and 103 both have a propensity score of 0.9 — their baseline characteristics strongly predict Drug A — but patient 101 received Drug A (treated) and patient 103 received Drug B (control). Patients 102 and 104 have a propensity score of 0.5, indicating true clinical equipoise. The analyst computes IPTW and overlap weights for all four patients and compares the resulting effective sample sizes to show why the extreme IPTW weight for patient 103 is problematic.

Dataset

Four-patient illustration cohort. e_hat is the estimated propensity score (probability of receiving Drug A). Arm 1 = Drug A (treated), Arm 0 = Drug B (control).

Steps

• IPTW weights for treated arm (weight = 1/e_hat). Patient 101 (treated, e = 0.9): IPTW = 1/0.9 = 10/9 = 1.11. Patient 102 (treated, e = 0.5): IPTW = 1/0.5 = 2.

• IPTW weights for control arm (weight = 1/(1 - e_hat)). Patient 103 (control, e = 0.9): IPTW = 1/(1-0.9) = 1/0.1 = 10 — a nine-fold disparity versus patient 101 who shares the same propensity score but is in the treated arm. Patient 104 (control, e = 0.5): IPTW = 1/(1-0.5) = 1/0.5 = 2. Patient 103 alone accounts for the bulk of the control pseudo-population, which means the IPTW estimator is largely determined by this one patient.

• Overlap weights for treated arm (weight = 1 - e_hat). Patient 101: w = 1-0.9 = 0.1. Patient 102: w = 1-0.5 = 0.5. High-PS treated patients receive small weights because they are far from the clinical equipoise region and contribute little to the ATO estimand.

• Overlap weights for control arm (weight = e_hat). Patient 103: w = 0.9. Patient 104: w = 0.5. All four overlap weights fall in [0.1, 0.9] — bounded by construction, no trimming required. The maximum IPTW weight was 10; the maximum overlap weight is 0.9.

• Overlap-weighted effective sample size. Sum of weights: 0.1+0.5+0.9+0.5 = 2.0. Sum of squared weights: 0.01+0.25+0.81+0.25 = 1.32. ESS = 2.0*2.0/1.32 = 3.03 effective patients out of 4. An IPTW analysis with weights 1.11, 2, 10, 2 would have ESS approximately 2.09 due to the extreme control weight of 10 dominating the denominator.

Result

IPTW creates a nine-fold weight disparity for the high-PS pair: treated weight = 1/0.9 = 10/9 = 1.11; control weight = 1/(1-0.9) = 1/0.1 = 10. Overlap weights for the same pair are bounded: treated = 1-0.9 = 0.1; control = 0.9. All four overlap weights lie in [0.1, 0.9] with no trimming needed by construction. Overlap-weighted ESS = 2.0*2.0/1.32 = 3.03 out of 4 patients, versus IPTW ESS of approximately 2.09. The ATO estimand concentrates statistical mass on the equipoise patients (e near 0.5) and down-weights the high-PS patients who are far from the overlap region where causal inference is most credible.

Runnable example

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from lifelines import CoxPHFitter

xcols = ["age", "sex", "cci", "ckd", "prior_hf", "prior_insulin", "prior_hosp", "prior_cost"]

# ── Stage 1 (outcome-blind design): estimate e(X) via near-unpenalized logistic regression ──
lr = LogisticRegression(max_iter=2000, C=1e6)
lr.fit(df[xcols], df["treated"])
df["e"] = np.clip(lr.predict_proba(df[xcols])[:, 1], 1e-6, 1 - 1e-6)

# ── Overlap weights (ATO): w = 1 - e for treated, w = e for control ──
# Bounded in (0, 1) by construction; no trimming needed.
df["w_ato"] = np.where(df["treated"] == 1, 1 - df["e"], df["e"])

# ── IPTW (ATE) for comparison: shows the weight explosion overlap weights avoid ──
p = df["treated"].mean()
df["w_iptw"] = np.where(df["treated"] == 1, p / df["e"], (1 - p) / (1 - df["e"]))
df["w_iptw_trunc"] = df["w_iptw"].clip(upper=df["w_iptw"].quantile(0.99))  # truncated IPTW

# ── Balance diagnostic: weighted SMD for each covariate in the PS model ──
def wsmd(x, trt, w):
    m1 = np.average(x[trt == 1], weights=w[trt == 1])
    m0 = np.average(x[trt == 0], weights=w[trt == 0])
    v1 = np.average((x[trt == 1] - m1) ** 2, weights=w[trt == 1])
    v0 = np.average((x[trt == 0] - m0) ** 2, weights=w[trt == 0])
    return abs(m1 - m0) / np.sqrt((v1 + v0) / 2)

smds = {c: wsmd(df[c].values, df["treated"].values, df["w_ato"].values) for c in xcols}
ess_ato  = df["w_ato"].sum()  ** 2 / (df["w_ato"]  ** 2).sum()
ess_iptw = df["w_iptw"].sum() ** 2 / (df["w_iptw"] ** 2).sum()
print(f"ATO  max |SMD| = {max(smds.values()):.3f} | ESS = {ess_ato:.0f} / {len(df)}")
print(f"IPTW max weight = {df['w_iptw'].max():.1f} | IPTW ESS = {ess_iptw:.0f}")
assert max(smds.values()) < 0.1, "overlap-weighted SMD > 0.1: revisit the PS model"

# ── Stage 2 (analysis): marginal weighted Cox model with robust (sandwich) SEs ──
# robust=True uses the Binder-type sandwich variance that accounts for estimated weights.
cox = CoxPHFitter()
cox.fit(
    df[["time", "event", "treated", "w_ato"]],
    duration_col="time", event_col="event",
    weights_col="w_ato", robust=True
)
print(cox.summary[["coef", "exp(coef)", "se(coef)", "p"]])
# exp(coef) = ATO hazard ratio; state the ATO estimand explicitly in all reporting.
# The ATO population is the clinical-equipoise subpopulation, NOT the full cohort.

library(PSweight); library(WeightIt); library(cobalt); library(survey)

xform <- treated ~ age + sex + cci + ckd + prior_hf + prior_insulin + prior_hosp + prior_cost

# ── Option A: PSweight — the canonical ATO package ──
# Estimates e(X) internally, constructs overlap weights, and provides the ATO point estimate
# with sandwich variance via a single call. The canonical reference for the PSweight package
# is the vignette at cran.r-project.org/package=PSweight.
ps_fit <- PSweight(
  ps.formula = xform,
  yname      = "event",        # binary or continuous outcome column name
  data       = df,
  weight     = "overlap"       # "overlap" = ATO; "IPW" = ATE; "treated" = ATT (SMR)
)
summary(ps_fit)                # ATO point estimate, 95% CI via sandwich SE

# ── Option B: WeightIt + cobalt — integrates with the broader WeightIt ecosystem ──
w <- weightit(xform, data = df, method = "ps", estimand = "ATO")
# estimand = "ATO" sets w = 1 - e for treated, w = e for control (overlap weights)
bal.tab(w, un = TRUE, thresholds = c(m = 0.1))   # SMDs pre and post overlap weighting
love.plot(w, abs = TRUE, thresholds = c(m = 0.1))
cat("ATO ESS:", summary(w)$effective.sample.size, "\n")

# ── Marginal weighted Cox model with robust (sandwich) SEs ──
df$w_ato <- w$weights
des <- svydesign(ids = ~1, weights = ~w_ato, data = df)
fit <- svycoxph(Surv(time, event) ~ treated, design = des)
summary(fit)   # exp(coef) = ATO hazard ratio; state the ATO estimand in all reporting

# ── Entropy balancing: ATT-style (control reweighted to match treated moments) ──
# method = "ebal" solves a convex optimization; no PS model estimated.
w_eb <- weightit(xform, data = df, method = "ebal", estimand = "ATT")
bal.tab(w_eb, un = TRUE, thresholds = c(m = 0.1))

# ── CBPS: PS estimated under dual fit-and-balance GMM constraint ──
w_cbps <- weightit(xform, data = df, method = "cbps", estimand = "ATO")
bal.tab(w_cbps, un = TRUE, thresholds = c(m = 0.1))
# Combine CBPS-estimated e(X) with overlap weight formula for robustness to PS misspecification