Individual Participant Data (IPD) Meta-Analysis

In plain language

Individual Participant Data (IPD) meta-analysis is a way to combine evidence from multiple studies by collecting the actual raw patient records from each study — not just the summary numbers published in a journal article. Having every patient's row of data lets researchers apply the same eligibility rules, the same covariate adjustments, and the same outcome definition across all studies, so the comparison is apples-to-apples rather than apples-to-oranges. It also unlocks questions a standard meta-analysis simply cannot answer, such as whether a drug works better in older patients or in patients with worse kidney function. The trade-off is real: gathering and harmonizing raw data from multiple institutions is expensive, requires data-sharing agreements, and is only feasible when data partners are willing to participate.

Individual participant data (IPD) meta-analysis

is the synthesis of the original patient-level records from multiple studies or data partners, rather than the published/aggregate summary statistics that conventional aggregate-data (AD) meta-analysis must rely on. Having the raw rows lets the analyst standardize eligibility, covariate definitions, follow-up time, and the outcome model across all contributors, then estimate a single pooled effect with quantified between-study heterogeneity. In real-world evidence it is the analytic backbone of distributed/multi-database programs (FDA Sentinel, the OHDSI network, CNODES), where each site runs an identical protocol on its own claims or EHR extract and the network combines the results.

Core estimand distinction

— IPD-MA targets the same pooled comparative effect (a log-HR, log-OR, or risk difference) as AD meta-analysis, but two structural choices define how it is estimated and what it can additionally recover. (1) Two-stage: fit the chosen model within each study (e.g., a per-study Cox model on the matched cohort), extract the study-specific effect and its standard error, then combine those estimates with a standard random-effects (DerSimonian–Laird or REML) or common-effect meta-analysis. (2) One-stage: fit one hierarchical model to the stacked patient rows, with study entered as a stratification/random-effect term so that nuisance parameters (baseline risk, baseline hazard) are study-specific while the treatment effect is shared or random across studies. The two can give different numbers — not because of a bug, but because they make different assumptions about how nuisance parameters are estimated, how the within- and between-study information is weighted, and ML vs REML estimation of the heterogeneity variance τ² (Burke, Ensor & Riley 2017). The decisive IPD-only capability is the unbiased estimation of within-study treatment–covariate interactions: pooling published subgroup effects conflates the within-study interaction (what a clinician needs) with the across-study ecological/aggregation association, whereas IPD separates them (the "deft" approach of Fisher et al. 2017). IPD-MA does not by itself remove confounding, publication bias, or study-level design flaws — it inherits whatever the contributing studies carry.

Pros, cons, and trade-offs

- vs aggregate-data (AD) meta-analysis of published effects: IPD lets you re-define the cohort identically everywhere, re-instate excluded covariates, reanalyze time-to-event with a common model, recover unbiased subgroup/interaction effects, and check individual-level assumptions (proportional hazards, functional form). Cost: it is enormously more expensive, requires data-sharing/governance, and demands serious harmonization labor. Prefer IPD when interactions, time-to-event reanalysis, or standardized confounding control are the point; AD is acceptable when the marginal main effect is all that is needed and IPD is unobtainable. - vs network meta-analysis (NMA): NMA connects many treatments through indirect comparisons but usually runs on aggregate arm-level data; IPD-MA is typically pairwise/few-treatment but patient-level. They are complementary — IPD-NMA exists and is the most demanding of all. Prefer IPD-MA for a focused head-to-head where interaction and patient-level adjustment matter. - vs a single pooled-rows analysis that ignores study (naive merge): simply concatenating everyone's rows and fitting one ordinary model ignores clustering by study, can induce Simpson's-paradox reversals, and produces falsely precise standard errors. One-stage IPD-MA fixes this by stratifying/randomizing the study term. Never do the naive merge. - two-stage vs one-stage within IPD-MA: two-stage is transparent, mirrors familiar forest-plot meta-analysis, and is the natural fit for federated analyses where only per-site summary statistics can leave the firewall. One-stage is more efficient with few or small studies and rare events, handles within-study interactions and non-linear terms more naturally, but is easier to misspecify (especially the τ² and the choice of common vs random treatment effect). Prefer two-stage when studies are large and governance blocks row-level pooling; prefer one-stage for sparse data, complex modeling, or interaction estimation.

When to use

— a focused comparative-effectiveness or safety question where the contributing studies/databases can supply patient-level data; distributed RWE networks running a common protocol; questions about who benefits (treatment-effect modification by age, renal function, baseline severity); time-to-event endpoints that the original papers reported only as crude rates; situations where standardizing covariate adjustment and eligibility across heterogeneous sources is essential for a credible pooled estimate.

When NOT to use — and when it is actively misleading or dangerous

- When only a handful of contributors will share data and they are the systematically "better" studies. Availability bias means the IPD subset is not the evidence base; an IPD-MA on the cooperative minority can be more biased than an AD-MA of everyone. Always report what fraction of eligible studies/participants supplied IPD and benchmark against the AD effect. - When the underlying studies are confounded or biased. IPD does not launder design flaws. A pooled confounded effect is a precise wrong answer. Harmonized confounding control (e.g., the same propensity model per site) is necessary but not sufficient if key confounders are unmeasured everywhere. - The naive single-model merge ignoring study. Treating all rows as one sample fabricates precision and can reverse the direction of the effect (aggregation/Simpson bias). - Reading an across-study (ecological) covariate association as an individual-level interaction. A trend in pooled per-study effects against mean study age is not the patient-level age interaction; using it to personalize treatment is exactly the deluded approach Fisher et al. warn against. - Forcing a common-effect model onto genuinely heterogeneous studies/databases. With real between-study τ² a common-effect pooled CI is far too narrow; conversely, estimating τ² from 2–3 studies is unstable and may need a Bayesian prior or a Hartung–Knapp adjustment.

Data-source operational depth

- Claims (FFS vs MA vs commercial): Each database defines exposure from the pharmacy claim (NDC + `fill_date` + `days_supply`) and requires continuous medical+pharmacy enrollment across the washout so "no prior fill" is observed rather than missing. Failure modes that differ by site and quietly drive heterogeneity: Medicare Advantage and capitated person-time lack fee-for-service claims (a site heavy in MA-only members has artificially low captured utilization — exclude MA-only person-time or model the difference), differential coding intensity between commercial and Medicare populations, and competing risks by death that vary with the age mix (an elderly Medicare site has more deaths censoring the event of interest). Harmonize the code lists and the continuous-enrollment rule centrally before any site runs the model. - EHR: Initiation is the order/administration, not a dispensing, and capture is visit-driven, so a patient who leaves the system is differentially lost. Sites differ in note/lab availability, so a covariate that exists at one site is missing at another — decide centrally whether such a covariate is dropped network-wide or handled by per-site imputation, because inconsistent handling masquerades as between-study heterogeneity. - Registry: Strong indication, severity, and adjudicated outcomes but typically incomplete pharmacy exposure; link to claims for fills and to a death index for censoring before contributing to the pool. - Linked claims–EHR–vital records: the ideal per-site substrate (severity + completeness + mortality), but the linkable subset is a selected population and order/fill/service-date discrepancies must be reconciled before time-zero assignment so that time-zero means the same thing at every site. - Federated vs centralized governance: when row-level data cannot leave a site (HIPAA, GDPR, payer contracts), run a two-stage design — each site fits the common model locally and exports only the coefficient and its SE (and, for an exact one-stage equivalent, sufficient statistics / score and information matrices). Beware immortal time introduced site-by-site in procedure or treatment-initiation cohorts: if any site starts follow-up before the exposure decision, it contributes a biased study-specific effect that the pooling step will faithfully carry forward.

Worked claims example (distributed two-stage IPD-MA)

Question: 1-year risk of hospitalized heart failure with a second-generation sulfonylurea vs a DPP-4 inhibitor among adults with type 2 diabetes, run across four data partners (a commercial claims plan, Medicare FFS, an integrated-delivery EHR linked to claims, and a regional registry linked to claims). A single protocol is distributed. At each site: (1) Eligibility — age ≥18, ≥2 diabetes diagnoses, and 365 days of continuous medical+pharmacy enrollment before the first study fill; exclude MA-only person-time so the washout is observable. (2) New-user washout — no fill of any sulfonylurea or DPP-4 inhibitor in the 365-day lookback. (3) Time zero — date of the first qualifying fill (`fill_date`); assign the arm from the dispensed NDC. (4) Confounding control — fit the same high-dimensional propensity score from covariates measured in `[index_date-365, index_date]` and apply 1:1 PS matching, checking standardized differences <0.1. (5) Outcome — first validated HF hospitalization; follow from time zero, censoring at disenrollment, death (mortality source hierarchy fixed centrally), end of data, and end of the 365-day risk window. (6) Stage 1 — each site fits a Cox model on its matched cohort and exports only the log-HR and its SE. (7) Stage 2 — the coordinating center pools the four log-HRs with a REML random-effects model, reporting the pooled HR, the 95% CI, and τ² / I² for between-site heterogeneity; a one-stage stratified Cox (baseline hazard stratified by site) is run as a sensitivity analysis on any sites that can share rows, and a meta-regression of the per-site HR on the site's MA-share and age mix probes whether the data-source failure modes above are driving the heterogeneity.

Worked example

Scenario

A research team wants to know whether Drug A lowers one-year hospitalization risk more than Drug B among adults with type 2 diabetes. Three separate database studies exist — a commercial claims study (Site 1), a Medicare claims study (Site 2), and an EHR-linked study (Site 3). A standard aggregate-data meta-analysis could combine only the three published hazard ratios. An IPD meta-analysis instead collects the raw patient rows from all three sites, harmonizes the data, and gains the ability to do things aggregate data cannot.

Dataset

Hypothetical patient rows that IPD meta-analysis works with (one row per patient, all three sites stacked after harmonization). An aggregate-data meta-analysis never sees these rows — it sees only three numbers.

Steps

• Each site receives the same analysis protocol specifying eligibility (age 18+, diagnosis codes, 365-day enrollment lookback), the same covariate list (age, HbA1c, comorbidities), and the same outcome definition (first hospitalization within 365 days).

• Each site harmonizes its data to the shared schema — the patient rows shown above represent what the combined, standardized dataset looks like after that step.

• Two-stage approach: each site fits a Cox survival model on its own matched cohort and reports back one log-hazard-ratio and one standard error (three numbers total from three sites).

• The coordinating center combines those three study-level estimates using a random-effects model (REML), producing a single pooled hazard ratio with a 95% confidence interval and a measure of how much the effect varied across sites (I-squared).

• Because the IPD contains each patient's age and baseline HbA1c, the team can also ask: does Drug A work better in patients with HbA1c above 9? This within-study interaction is estimated at each site and then pooled — an analysis that aggregate-data meta-analysis cannot perform correctly.

• Result: pooled HR = 0.78 (95% CI 0.65-0.93), I-squared = 18% (low heterogeneity). The subgroup analysis shows a stronger benefit in patients with HbA1c > 9.0 (HR 0.62), a finding only recoverable from the patient-level data.

Result

Pooled HR for hospitalization, Drug A vs Drug B = 0.78 (95% CI 0.65-0.93). The IPD approach also reveals that patients with baseline HbA1c above 9.0 benefit more (HR 0.62 in that subgroup) — a finding that would have been invisible to aggregate-data meta-analysis because published papers did not report that specific subgroup split.

Comparison Table

What each approach can and cannot do

Runnable example

import numpy as np
import pandas as pd
from lifelines import CoxPHFitter
import statsmodels.api as sm

def two_stage_ipd_ma(ipd: pd.DataFrame) -> dict:
    # ---- Stage 1: per-study Cox model, keep treatment log-HR and its SE ----
    rows = []
    for study, g in ipd.groupby("study_id"):
        if g["event"].sum() == 0 or g["treat"].nunique() < 2:
            continue  # uninformative site (no events or single-arm) - report, do not impute
        cph = CoxPHFitter().fit(g[["time", "event", "treat"]],
                                duration_col="time", event_col="event")
        rows.append({"study_id": study,
                     "loghr": cph.params_["treat"],
                     "se":    cph.standard_errors_["treat"]})
    stage1 = pd.DataFrame(rows)

    # ---- Stage 2: REML random-effects pool of the per-study log-HRs ----
    y, s2 = stage1["loghr"].to_numpy(), stage1["se"].to_numpy() ** 2
    tau2 = 0.0
    for _ in range(100):  # REML fixed-point iteration for between-study variance
        w = 1.0 / (s2 + tau2)
        mu = np.sum(w * y) / np.sum(w)
        tau2_new = max(0.0, (np.sum(w**2 * ((y - mu) ** 2 - s2)) +
                             (1.0 / np.sum(w))) / np.sum(w**2))
        if abs(tau2_new - tau2) < 1e-10:
            break
        tau2 = tau2_new
    w = 1.0 / (s2 + tau2)
    mu = np.sum(w * y) / np.sum(w)
    se_mu = np.sqrt(1.0 / np.sum(w))
    Q = np.sum((y - np.sum((y / s2)) / np.sum(1 / s2)) ** 2 / s2)
    I2 = max(0.0, (Q - (len(y) - 1)) / Q) if Q > 0 else 0.0
    return {"per_study": stage1, "pooled_HR": np.exp(mu),
            "ci95": (np.exp(mu - 1.96 * se_mu), np.exp(mu + 1.96 * se_mu)),
            "tau2": tau2, "I2": I2}

def one_stage_ipd_ma(ipd: pd.DataFrame):
    # Stratified Cox: study-specific baseline hazard (nuisance), shared treatment log-HR.
    cph = CoxPHFitter()
    cph.fit(ipd[["time", "event", "treat", "study_id"]],
            duration_col="time", event_col="event", strata=["study_id"])
    return cph  # cph.summary holds the pooled treat log-HR, SE, CI

# res = two_stage_ipd_ma(ipd); print(res["pooled_HR"], res["ci95"], res["I2"])

library(survival)
library(metafor)

# ---- Stage 1: per-study Cox model, collect treatment log-HR and SE ----
stage1 <- do.call(rbind, lapply(split(ipd, ipd$study_id), function(g) {
  if (sum(g$event) == 0 || length(unique(g$treat)) < 2) return(NULL)  # uninformative site
  fit <- coxph(Surv(time, event) ~ treat, data = g)
  data.frame(study_id = g$study_id[1],
             loghr = coef(fit)[["treat"]],
             se    = sqrt(vcov(fit)[["treat", "treat"]]))
}))

# ---- Stage 2: REML random-effects pool (Hartung-Knapp when few studies) ----
pooled <- rma(yi = stage1$loghr, sei = stage1$se,
              method = "REML", test = "knha")
pooled_HR <- exp(pooled$b)                              # pooled hazard ratio
ci        <- exp(c(pooled$ci.lb, pooled$ci.ub))         # 95% CI
c(HR = pooled_HR, lcl = ci[1], ucl = ci[2],
  tau2 = pooled$tau2, I2 = pooled$I2)

# ---- One-stage alternative: study-stratified Cox (study-specific baseline hazard) ----
one_stage <- coxph(Surv(time, event) ~ treat + strata(study_id), data = ipd)
summary(one_stage)$coefficients   # pooled treat log-HR, SE, CI from the stacked rows