Calendar-Time Bias and Secular-Trend Confounding

Calendar-time bias

and secular-trend confounding occur when the clock on the wall is part of the data-generating mechanism. RWE studies often combine patients initiated across years during which indications broaden, prescribing restrictions change, generics enter, safety warnings appear, diagnostic codes migrate, care pathways improve, data feeds mature, and outcome incidence shifts. If the exposed group is concentrated in one calendar period and the comparator group in another, a treatment contrast can become a disguised before-versus-after comparison.

Calendar time is not always a confounder. It becomes one when it predicts treatment assignment and outcome risk or outcome capture. The problem is especially sharp during product lifecycles. Early adopters of a newly approved therapy may be younger, privately insured, treated at academic centers, or selected for better prognosis; later users may include frailer or contraindication-heavy patients after guidelines broaden. Conversely, a safety warning can channel high-risk patients away from a drug, making later users healthier than earlier users. A conventional propensity score estimated over the full study period can average over those changing predictors and leave within-period imbalance, even if an overall year indicator is included.

Mechanisms in RWE

Calendar-time bias enters through treatment diffusion, coding shifts, data source evolution, and outcome-risk trends. Treatment diffusion includes drug launch, label expansion, generic entry, guideline endorsement, prior authorization, shortage, and safety communication. Coding shifts include ICD-9 to ICD-10 migration, new procedure codes, EHR template adoption, value- based quality reporting, and improved capture of laboratory or mortality feeds. Outcome-risk trends include background improvements in survival, pandemic disruptions, screening uptake, changes in competing mortality, and new co-interventions. A naive model that pools all initiators and adjusts for "year" can still fail if the treatment assignment model itself changes by year.

Pros, cons, and trade-offs

Calendar restriction, calendar matching, and calendar-time-specific propensity scores protect exchangeability by comparing patients initiated under similar practice conditions. The cost is smaller risk sets and weaker positivity: in early launch periods there may be many treated patients but few comparable active-comparator initiators, or the reverse. Calendar fixed effects are simple and often necessary, but they do not guarantee covariate balance within each period. Staggered difference-in-differences and interrupted time series can estimate policy effects when the exposure is a calendar-timed intervention, but they require parallel-trend or counterfactual-trend assumptions and are not substitutes for individual-level confounding control in drug initiation cohorts.

When NOT to use — and when it is actively misleading

Do not force fine calendar matching when the treatment is rare and positivity collapses; exact month matching can exchange confounding for severe extrapolation if one arm barely exists in some months. Do not adjust for calendar time only after it has been affected by the exposure, such as conditioning on post-launch healthcare contacts that are downstream of the new therapy's monitoring program. Do not use historical controls for a rapidly improving outcome without a design that directly models the secular trend. Do not interpret a pre-post change as a treatment effect when coding, surveillance, or background therapy changed at the same time as treatment adoption.

Data-source operational depth

• Claims: Enrollment mix, payer contracts, formulary rules, NDC availability, reversals, and benefit carve-outs can change by month. The absence of a code or fill in 2012 may not mean the same thing as absence in 2024. Build arm-by-month initiation plots, require contemporaneous active comparators where possible, and avoid pooling FFS and Medicare Advantage observable time across periods without explicit completeness checks.
• EHR: Template changes, new order sets, lab interface rollouts, telehealth adoption, and site onboarding can change measurement independently of disease. Use site-specific calendar controls when EHR adoption is staggered and check whether missingness, lab frequency, and outcome coding move discontinuously at go-live dates.
• Registry: Registries mature: early years can be referral-center heavy with incomplete follow-up, while later years include broader sites and more complete mortality linkage. For registry-based external controls, align diagnosis, treatment, and follow-up periods to the intervention cohort or model calendar trends explicitly.
• Linked data: Linked claims-EHR-registry datasets can have source-specific refresh lags. Calendar time must be defined separately for treatment initiation, outcome occurrence, claim processing, EHR extraction, registry abstraction, and death-index refresh.

Decision diagram

flowchart TD
  CT["Calendar period"]
  TX["Treatment choice"]
  O["Outcome risk / outcome capture"]
  X["Measured patient risk"]
  Y["Observed outcome"]

  CT --> TX
  CT --> O
  X --> TX
  X --> Y
  TX --> Y
  O --> Y

Worked example

Scenario

A new oncology drug enters practice in 2021. The comparator cohort is mostly treated in 2018-2019, before a survival-improving supportive-care guideline and before registry mortality linkage improved. A pooled model estimates better survival for the new drug.

Dataset

Arm distribution and background risk by calendar period.

Steps

Result

The crude new-drug survival advantage may be a calendar-period advantage. A credible analysis uses contemporaneous comparators, calendar-time-specific propensity scores, or a design for historical controls that explicitly models the secular survival trend.

Trade-offs

Runnable example

import numpy as np
import pandas as pd
import statsmodels.api as sm

COVARS = ["age", "female", "charlson", "prior_hosp", "prior_rx_count"]

def fit_period_ps(df: pd.DataFrame) -> pd.DataFrame:
    pieces = []
    for period, d in df.groupby("period"):
        if d["treated"].nunique() < 2:
            continue
        x = sm.add_constant(d[COVARS], has_constant="add")
        ps = sm.Logit(d["treated"], x).fit(disp=False).predict(x)
        tmp = d.copy()
        tmp["ps"] = ps.clip(0.01, 0.99)
        tmp["att_weight"] = np.where(tmp["treated"] == 1, 1.0, tmp["ps"] / (1 - tmp["ps"]))
        pieces.append(tmp)
    return pd.concat(pieces, ignore_index=True)

analytic = fit_period_ps(cohort)

# Positivity diagnostic: treatment and comparator must overlap within each period.
overlap = analytic.groupby(["period", "treated"])["ps"].agg(["min", "median", "max", "count"])
print(overlap)

library(data.table)
setDT(cohort)
covars <- c("age", "female", "charlson", "prior_hosp", "prior_rx_count")

fit_one <- function(d) {
  if (length(unique(d$treated)) < 2L) return(NULL)
  f <- as.formula(paste("treated ~", paste(covars, collapse = " + ")))
  m <- glm(f, data = d, family = binomial())
  d[, ps := pmin(pmax(predict(m, type = "response"), 0.01), 0.99)]
  d[, att_weight := fifelse(treated == 1L, 1, ps / (1 - ps))]
  d
}

analytic <- cohort[, fit_one(.SD), by = period]
print(analytic[, .(min_ps = min(ps), med_ps = median(ps),
                   max_ps = max(ps), n = .N), by = .(period, treated)])

proc sort data=work.cohort out=cohort_s;
  by period;
run;

proc logistic data=cohort_s descending noprint;
  by period;
  model treated = age female charlson prior_hosp prior_rx_count;
  output out=ps_out pred=ps_raw;
run;

data analytic;
  set ps_out;
  ps = min(max(ps_raw, 0.01), 0.99);
  if treated = 1 then att_weight = 1;
  else att_weight = ps / (1 - ps);
run;

proc means data=analytic min median max n;
  class period treated;
  var ps;
run;

Citations

1. [1]Mack CD, Glynn RJ, Brookhart MA, Carpenter WR, Meyer AM, Sandler RS, Stürmer T. Calendar time-specific propensity scores and comparative effectiveness research for stage III colon cancer chemotherapy. Pharmacoepidemiology and Drug Safety. 2013;22(8):810-818.
2. [2]Petri H, Urquhart J. Channeling bias in the interpretation of drug effects. Statistics in Medicine. 1991;10(4):577-581.

1. [3]Dusetzina SB, Mack CD, Stürmer T. Propensity Score Estimation to Address Calendar Time-Specific Channeling in Comparative Effectiveness Research of Second Generation Antipsychotics. PLoS ONE. 2013;8(5):e63973.

1. [4]Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology. 2016;183(8):758-764.