Multi-Database / Distributed Network Study

In plain language

A multi-database study runs the exact same research question across several independent health databases at the same time, combines the findings from each one, and checks whether they agree. No patient records are ever shared between databases — each site keeps its own data, runs the same computer code locally, and sends back only a privacy-safe summary number. Running the same study across multiple sources boosts statistical power for rare events, shows whether a finding holds across different patient populations, and lets researchers spot when one database tells a very different story from the others.

A multi-database (distributed network) study answers a single research question by executing one harmonized protocol across several independently governed data sources — commercial and Medicare claims plans, integrated-delivery EHRs, registries — and then combining the results, not the records. Each data partner maps its data to a shared structure (a common data model such as OMOP CDM or the Sentinel SCDM, or a shared analytic-table specification), runs byte-identical or specification-identical code locally, and returns only aggregate output. The patient-level protected health information never leaves the partner's firewall. This is the operating model of the FDA Sentinel System, the Canadian Network for Observational Drug Effect Studies (CNODES), the OHDSI network, and most multinational PASS.

Core conceptual distinction

The defining choice is distributed analysis vs centralized pooling, and it is separable from the choice of estimator. (1) Distributed vs pooled individual-level data (IPD): in a distributed network each site holds its own data and shares only aggregates; in a pooled study all records are physically combined in one analytic file. Distributed analysis is what makes data partners willing to participate (governance, HIPAA, GDPR), but it constrains what you can compute — you must design every quantity so it can be assembled from site-level pieces. (2) Common data model vs ad hoc harmonization: a CDM (OMOP, SCDM/Sentinel) fixes table schemas and vocabularies so the same code runs everywhere; ad hoc harmonization writes bespoke extraction per site and is fragile. (3) What gets shared sits on a spectrum: fully aggregate counts/effect estimates (most private), stratified risk-set or propensity-score-stratum tables that enable exact stratified or conditional analyses without IPD (Toh's distributed risk-set sharing), or, rarely, a curated pooled extract under a data-use agreement. The estimand is still a comparative effect (e.g., a hazard ratio or risk difference for drug A vs B), but it is a network-level summary of site-specific estimates whose interpretation depends on whether you fix-effect or random-effect combine them, and on how heterogeneous the sites are.

Pros, cons, and trade-offs

- vs a single-database study: A network buys sample size for rare exposures and outcomes, broader generalizability across payers/regions/care settings, and the ability to measure heterogeneity (is the signal real or one site's artifact?). Cost: enormous operational overhead — common-protocol governance, CDM mapping, code distribution, site QC, and a slower timeline. Prefer a network when one database lacks power, when external validity matters for a regulatory or coverage decision, or when reproducibility across data sources is itself the evidentiary claim; prefer a single well-characterized database for an exploratory or hypothesis-generating analysis where speed and deep knowledge of one source outweigh breadth. - vs centralized pooled individual-level analysis: Distributed analysis preserves privacy and partner autonomy and sidesteps the legal/ethical barriers to moving PHI. Cost: you cannot run arbitrary individual-level models centrally; you are limited to what aggregates support (stratified Cox/conditional logistic via risk sets, site-specific PS models, meta-analytic combination). Some flexible estimators (certain machine-learning PS, individual-level g-methods) are awkward or impossible without IPD. Prefer distributed unless a data-use agreement genuinely permits pooling and the analysis demands individual-level modeling that aggregates cannot reproduce. - vs aggregate meta-analysis of separately published studies: A distributed network uses one protocol, one set of code lists, and one outcome definition everywhere, eliminating the between-study methodological heterogeneity and publication bias that plague literature-based meta-analysis. Cost: it requires a live, governed network rather than a desk review. Prefer the network when you control the analysis; fall back to meta-analysis of published estimates only when you cannot access the underlying data.

When to use

Use a multi-database design when (a) no single source is large enough for the exposure-outcome pair (rare drugs, rare adverse events, subgroups); (b) a regulator (FDA Sentinel, EMA PASS) or HTA body expects evidence reproduced across multiple real-world sources; (c) generalizability across populations, payers, or health systems is central to the decision; or (d) consistency across heterogeneous data is itself the scientific point (does the effect replicate?). It is the right substrate for active drug-safety surveillance and for high-stakes comparative safety questions where a single-database finding would not be persuasive.

When NOT to use — and when it is actively misleading or dangerous

- When one database already answers the question with adequate power and validity. The network's overhead buys nothing and slows the answer; the breadth is decorative. - When the exposure, outcome, or confounders are not measurable identically across sites. If inpatient drug administration is captured at one integrated-delivery site but invisible in another's claims, the "same" exposure definition means different things; combining them manufactures a spurious network estimate. Pooling or meta-analyzing non-comparable site definitions is the dangerous failure mode — heterogeneity then reflects measurement, not biology, and a fixed-effect summary will confidently report a number that means nothing. - When sites differ structurally in ways the analysis ignores. Different drug launch dates and formulary timing by plan create calendar-time confounding that varies by site; Medicare Advantage vs fee-for-service capture differs; case-mix differs. Blindly fixed-effect combining hides this. Always report between-site heterogeneity (I², τ², forest plot) and investigate large I² before trusting a pooled number. - When privacy constraints force aggregates so coarse that the estimator is biased. If a site can only return marginal counts (not PS-stratified risk sets), residual confounding cannot be controlled distributedly and the network estimate is no better than a crude one.

Data-source operational depth

- Claims (commercial / Medicare FFS): The workhorse of Sentinel and CNODES. Exposure = pharmacy claim (`ndc` + `fill_date` + `days_supply`); enrollment spans define observable time. Require continuous medical + pharmacy enrollment across washout and follow-up so absence of a fill is real. Failure mode: Medicare Advantage person-time lacks FFS claims — MA enrollees' encounters are paid by the plan, not adjudicated as FFS claims, so diagnoses/procedures are missing or undercounted; exclude MA-only person-time (or use MA encounter data only where a partner certifies its completeness). Sample fills, 90-day mail order, and free samples distort `days_supply`. - EHR / integrated delivery (e.g., a Sentinel data partner with an internal pharmacy): Captures inpatient administrations, labs, and vitals that claims miss — an advantage that becomes a threat to comparability when pooled with claims-only partners. A drug given in hospital is observed at the EHR site and invisible at the claims site, so the operational exposure definition silently differs. Visit-driven capture also means patients who leave the system are differentially lost. Workaround: restrict to the lowest common denominator of captured care, or model site as a fixed effect and stratify so each site's estimate uses only its own internally consistent data. - Registry: Strong for indication, disease severity, and adjudicated outcomes (e.g., cancer stage, validated MI); weak for complete longitudinal drug exposure. Use registries in a network for the outcome/severity layer and link to claims for exposure and to a death index for censoring; never assume a registry's drug history is complete. - Linked claims–EHR–vital records: The richest partner type, but linkage selects the linkable subset and creates order/fill/service date discrepancies that must be reconciled before time-zero assignment. In a network, a few linked sites plus many claims-only sites create a capability gradient — design the common protocol to the weakest site, then run richer sensitivity analyses only where the data support them. - Cross-cutting failure modes: differential competing risks by exposure (in elderly claims populations a drug preferentially used in frailer patients faces higher competing mortality, biasing cause-specific estimates differently across sites with different age mixes); immortal time in procedure studies (defining exposure by a procedure that can only occur after surviving to it); and outcome-algorithm portability (a claims-based MI algorithm validated in one plan may have different PPV in another).

Worked claims example (distributed safety study)

Question: incidence of acute pancreatitis among new users of incretin-based therapy (GLP-1/DPP-4) vs sulfonylureas, run across four data partners (two commercial claims plans, one Medicare FFS extract, one integrated-delivery EHR), one common protocol. At each site, local code builds the analytic table identically: (1) Cohort entry = first fill (`fill_date`) of either drug class with no fill of any study or comparator drug in the prior 365 days (washout), among adults with ≥2 type-2-diabetes diagnoses in the baseline window. (2) Observable time = continuous medical + pharmacy enrollment spanning the full 365-day washout through follow-up; exclude MA-only person-time at the Medicare partner because FFS claims are absent there. (3) Index date / time zero = the qualifying fill date; assign the arm from the `ndc` dispensed that day. (4) Outcome = first inpatient acute-pancreatitis diagnosis (validated algorithm) after time zero; censor at disenrollment, death (death index), end of data, treatment discontinuation (`days_supply` end + 30-day grace), or switch. (5) Each site estimates a site-specific propensity score from baseline covariates measured only in `[index_date-365, index_date]`, forms PS strata, and returns only a stratified risk-set / event-count table per PS stratum and arm (person-time and events) — no patient-level rows leave the site. (6) The coordinating center combines the four site-specific stratified incidence-rate ratios with a random-effects meta-analysis, reports the pooled IRR with its 95% CI, and reports I² and a forest plot; a high I² triggers investigation of whether one partner's inpatient capture or formulary timing — not biology — drives the divergence before any pooled number is released.

Worked example

Scenario

A research team wants to know whether GLP-1 diabetes medications are associated with fewer hospitalizations for pancreatitis compared with sulfonylureas. No single insurance database has enough pancreatitis cases to answer this reliably, so the team runs the identical study at three separate data partners. Each partner maps its data to a shared common data model, runs the same code locally, and returns only a small table of event counts and patient-years. The coordinating center then pools those three site-level estimates.

Dataset

Privacy-preserving summary table returned by each data partner (no patient rows cross the firewall). events_study = pancreatitis hospitalizations in GLP-1 arm; PY_study = patient-years in GLP-1 arm; events_comp = hospitalizations in sulfonylurea arm; PY_comp = patient-years in sulfonylurea arm.

Steps

• For each site, compute the incidence rate ratio (IRR): divide the GLP-1 event rate (events_study / PY_study) by the sulfonylurea event rate (events_comp / PY_comp).

• Site A IRR = (3 / 600) / (6 / 600) = 0.005 / 0.010 = 0.500 — GLP-1 users had half the pancreatitis rate of sulfonylurea users at this site.

• Site B IRR = (5 / 800) / (8 / 800) = 0.00625 / 0.01000 = 0.625 — GLP-1 users had 62.5% of the sulfonylurea rate at this site.

• Site C IRR = (6 / 1000) / (8 / 1000) = 0.006 / 0.008 = 0.750 — GLP-1 users had 75% of the sulfonylurea rate at this site.

• Pool the three site-specific IRRs by taking a simple average: (0.500 + 0.625 + 0.750) / 3 = 1.875 / 3 = 0.625.

• Note the spread across sites: IRRs range from 0.500 to 0.750, which signals real heterogeneity. Before releasing the pooled number, the coordinating center investigates whether one site captures inpatient events differently or serves a different age mix.

Result

Pooled IRR = (0.500 + 0.625 + 0.750) / 3 = 1.875 / 3 = 0.625. Across all three data partners, GLP-1 users had approximately 37.5% fewer pancreatitis hospitalizations per patient-year than sulfonylurea users (IRR 0.625). The finding is consistent in direction across all three sites, which strengthens confidence in the signal. The range of site-specific estimates (0.500 to 0.750) is worth reporting so readers can judge how much the result varies by data source.

Runnable example

import pandas as pd
import numpy as np

WASHOUT_DAYS = 365   # drug-free + continuous-enrollment lookback defining a new user
GRACE_DAYS   = 30    # as-treated grace period after last days_supply
MIN_CELL     = 11    # suppress small cells before sharing (re-identification guard)

def site_summary(rx: pd.DataFrame, enroll: pd.DataFrame,
                 events: pd.DataFrame, cov: pd.DataFrame, data_partner_id: str) -> pd.DataFrame:
    rx = rx.sort_values(["person_id", "fill_date"])
    study = rx[rx["drug_class"].isin(["STUDY", "COMPARATOR"])]

    # New-user index: first qualifying fill; arm = drug_class dispensed that day.
    idx = (study.groupby("person_id").first().reset_index()
                .rename(columns={"fill_date": "index_date", "drug_class": "arm"}))

    # Washout: drop anyone with a prior study/comparator fill in the 365 days before index.
    prior = study.merge(idx[["person_id", "index_date"]], on="person_id")
    bad = prior[(prior["fill_date"] < prior["index_date"]) &
                (prior["fill_date"] >= prior["index_date"] - pd.Timedelta(days=WASHOUT_DAYS))]["person_id"]
    idx = idx[~idx["person_id"].isin(bad)].copy()

    # Continuous, FFS-observable enrollment across washout through index (no MA-only person-time).
    e = enroll.merge(idx[["person_id", "index_date"]], on="person_id")
    e["ok"] = ((e["enroll_start"] <= e["index_date"] - pd.Timedelta(days=WASHOUT_DAYS)) &
               (e["enroll_end"]   >= e["index_date"]) & (~e["ma_only"]))
    idx = idx[idx["person_id"].isin(e.loc[e["ok"], "person_id"])].copy()

    # As-treated exit: min(last days_supply end + grace, end of enrollment).
    last_supply = (study.merge(idx[["person_id"]], on="person_id")
                        .assign(supply_end=lambda d: d["fill_date"] + pd.to_timedelta(d["days_supply"], "D"))
                        .groupby("person_id")["supply_end"].max())
    enr_end = enroll.groupby("person_id")["enroll_end"].max()
    c = idx.merge(last_supply.rename("supply_end"), on="person_id") \
           .merge(enr_end.rename("enr_end"), on="person_id") \
           .merge(events.groupby("person_id")["event_date"].min().rename("event_date"),
                  on="person_id", how="left") \
           .merge(cov[["person_id", "ps_stratum"]], on="person_id", how="left")
    c["tx_exit"]  = (c["supply_end"] + pd.Timedelta(days=GRACE_DAYS)).clip(upper=c["enr_end"])
    c["exit"]     = c[["tx_exit", "enr_end"]].min(axis=1)
    had_event     = c["event_date"].notna() & (c["event_date"] <= c["exit"])
    c["exit"]     = np.where(had_event, c["event_date"], c["exit"])
    c["event"]    = had_event.astype(int)
    c["pt_days"]  = (pd.to_datetime(c["exit"]) - c["index_date"]).dt.days.clip(lower=0)

    # Aggregate to PS-stratum x arm; this is the ONLY thing that leaves the site.
    agg = (c.groupby(["ps_stratum", "arm"])
             .agg(events=("event", "sum"), person_years=("pt_days", lambda s: s.sum() / 365.25),
                  n=("person_id", "size")).reset_index())
    agg["data_partner_id"] = data_partner_id
    agg.loc[agg["n"] < MIN_CELL, ["events", "person_years", "n"]] = np.nan  # cell suppression
    return agg

library(data.table)
library(metafor)

pool_network <- function(sites) {
  setDT(sites)
  sites <- sites[!is.na(events) & !is.na(person_years)]   # drop suppressed cells

  # Site-specific log-IRR via PS-stratified Mantel-Haenszel (study vs comparator).
  w <- dcast(sites, data_partner_id + ps_stratum ~ arm,
             value.var = c("events", "person_years"))
  setnames(w, c("events_STUDY","events_COMPARATOR","person_years_STUDY","person_years_COMPARATOR"),
           c("e1","e0","pt1","pt0"))
  w <- w[pt1 > 0 & pt0 > 0]
  site <- w[, {
    tot <- e1 + e0; pt <- pt1 + pt0                       # stratum totals
    num <- sum(e1 * pt0 / pt); den <- sum(e0 * pt1 / pt)  # MH rate-ratio components
    v   <- sum((e1 + e0) * pt1 * pt0 / pt^2) / (num * den)
    .(logirr = log(num / den), se = sqrt(v))
  }, by = data_partner_id]

  # Random-effects (DerSimonian-Laird) pooling across sites + heterogeneity.
  m <- rma(yi = site$logirr, sei = site$se, method = "DL")
  list(IRR = exp(m$beta[[1]]),
       CI  = exp(c(m$ci.lb, m$ci.ub)),
       I2  = m$I2, tau2 = m$tau2,
       per_site = site[, .(data_partner_id, IRR = exp(logirr))])
}