Network Meta-Analysis

In plain language

Network meta-analysis is a method for comparing treatments that were never tested against each other head-to-head in a single trial. It works by chaining together results from multiple trials that share a common comparator — for example, if Drug A and Drug C were each tested against Drug B in separate trials, you can use the two sets of results to estimate how A and C compare to each other. The result is a single, coherent table of how every treatment in the network stacks up against every other, even for pairs that no single trial ever directly compared. The key risk is that this chaining only holds up if the patients in the different trials were similar enough that the shared comparator means the same thing in each study.

Network meta-analysis (NMA)

— also called mixed-treatment comparison (MTC) — generalizes pairwise meta-analysis to three or more interventions linked in a connected network. It borrows strength across the whole evidence base: a contrast between treatments B and C can be estimated even when no head-to-head B-vs-C trial exists, by chaining through a common comparator (B vs A and C vs A imply an indirect B-vs-C effect). When both direct and indirect evidence are present, NMA produces a mixed estimate that is a precision-weighted blend of the two. The output is a coherent set of all pairwise relative effects (a "league table") on one scale — log odds ratio, log hazard ratio, mean difference — plus a probabilistic ranking of treatments. In HEOR it is the standard tool for populating cost-effectiveness models when the manufacturer's drug was never trialled directly against the relevant comparators that a payer cares about.

Core estimand distinction

runs as follows. The estimand is the set of relative treatment effects d_XY for every pair (X, Y) of treatments in the network, anchored to a reference treatment so that d_XY = d_AY − d_AX (the consistency equation). This is fundamentally a synthesis of between-arm contrasts within randomized trials — NMA preserves within-study randomization and never compares a treatment arm in one trial to an arm in a different trial (that would discard randomization and reduce to a naive indirect comparison). Two parameterizations exist: the contrast-based Lu–Ades formulation (model the observed within-trial contrasts; the default in `netmeta`, `gemtc`, and most NICE DSU code) and the arm-based formulation (model arm-level means with trial random effects; estimates absolute risks but leans on stronger missing-at-random assumptions across the network). Effects can be fixed (one true effect per comparison) or random with a common heterogeneity variance τ² shared across comparisons — the near-universal default, because trials of different comparisons rarely share a single true effect. The Bucher adjusted indirect comparison is the special case of NMA that estimates a single anchored indirect comparison (B vs C) through one common comparator A, with no closed loops; NMA is its multi-arm, multi-loop generalization.

The two load-bearing assumptions

(1) Transitivity (a clinical/epidemiological assumption): effect modifiers — disease severity, age, prior lines of therapy, placebo response, definition and timing of the outcome — are distributed similarly across the trial sets that inform each comparison, so that the common comparator is genuinely exchangeable across loops. Transitivity is not testable from the synthesis data; it is defended by tabulating trial-level characteristics across comparisons. (2) Consistency is the statistical manifestation of transitivity: direct and indirect estimates of the same contrast agree. It is checkable in any network containing closed loops, via node-splitting (separate indirect from direct evidence for a contrast and test the difference), the design-by-treatment interaction test, or net-heat/net-splitting plots. A connected but loop-free ("tree" or "star") network can never have its consistency assessed — a critical and frequently missed limitation.

Pros, cons, and trade-offs

are specific and comparative below. - vs pairwise meta-analysis: NMA estimates contrasts with no direct trials and yields a single coherent ranking; it gains precision by borrowing strength. Cost: it imports the transitivity assumption and can be destabilized by one inconsistent loop or one influential small trial. Prefer pairwise when adequate head-to-head trials exist for the one contrast you care about — do not network just to network. - vs Bucher anchored indirect comparison: NMA handles >3 treatments, multi-arm trials (with the correct within-trial correlation), and mixed evidence in one model. Bucher is transparent and auditable for a single A-anchored B-vs-C contrast. Prefer Bucher for a simple two-trial indirect comparison a reviewer can replicate by hand; prefer NMA for any real network. - vs population-adjusted indirect comparison (MAIC / STC): When the anchoring assumption of a standard NMA fails because effect modifiers differ across trials — and you have individual patient data (IPD) for at least one trial — MAIC/STC re-weights or regresses to the comparator population. NMA assumes balance you cannot fix; MAIC/STC fixes imbalance you can measure but burns degrees of freedom and (in the unanchored case) makes the far stronger assumption that all prognostic factors are adjusted. Prefer population adjustment when transitivity is clearly violated and IPD exists; prefer NMA when the network is large and transitivity is plausible. - fixed vs random effects: random effects are honest about between-trial heterogeneity but estimate τ² poorly in sparse networks, inflating credible intervals and ranking instability; fixed effects understate uncertainty if heterogeneity is real. Report both and a prediction interval.

When to use

(decision rules). A connected network of randomized trials (RWE-derived effect estimates can be folded in, see below) where the decision-relevant comparison lacks adequate direct evidence; HTA submissions (NICE, CADTH, IQWiG) where a new drug must be compared to all relevant standards of care to populate a cost-effectiveness model; comparative-effectiveness questions across a therapeutic class with a shared reference (e.g., placebo or an old standard). Transitivity must be defensible and the network connected.

When NOT to use — and when it is actively misleading or dangerous

(decision rules). - Disconnected network. If the treatment of interest shares no comparator path with the target comparator, no amount of modelling connects them. Reporting an effect across a disconnected gap is fabrication, not synthesis. - Transitivity clearly violated. If trials of comparison A-vs-B enrolled first-line milder patients while A-vs-C trials enrolled refractory patients, the common comparator A is not exchangeable and the indirect C-vs-B estimate is confounded by the effect modifier — a structurally biased number presented with false precision. This is the single most dangerous failure mode; defend transitivity before you fit anything. - Loop-free network presented as validated. A star network around placebo cannot have its consistency tested. Claiming a "consistency-checked" NMA on such a network is misleading. - Over-reliance on rankings (SUCRA / P-score / rankograms). Ranking statistics compress the entire joint distribution into one number and routinely crown treatments whose effects are statistically indistinguishable from rivals, especially in sparse networks. A top SUCRA with a wide credible interval is rank instability, not superiority. Never report a rank without the underlying relative effects and uncertainty. - Dose or formulation lumping/splitting. Treating different doses of the same drug as one node hides a dose-response signal; splitting every dose into its own node fractures the network and inflates apparent heterogeneity. Pre-specify the node definition.

Data-source operational depth

follows. Classical NMA synthesizes published aggregate trial data — arm-level events/N (binary) or mean/SD/N (continuous), or pre-computed contrasts with standard errors fed through generic inverse-variance. The same machinery underpins RWE/HTA work but with source-specific failure modes: - Aggregate published RCT data: the common substrate. Failure modes: selective-outcome and selective-trial reporting open the network asymmetrically; multi-arm trials must contribute all pairwise contrasts with their induced within-trial covariance (ignoring it double-counts the shared arm and falsely narrows intervals); zero-event arms need continuity handling or an exact likelihood (`netmeta` GLM / Bayesian binomial) rather than a normal-approximation fudge. - IPD-NMA (individual patient data): when IPD is available for some trials, arm-level covariate interactions can be modelled to relax transitivity (meta-regression within the network). Failure mode: mixing IPD and aggregate trials risks ecological bias — an aggregate-level covariate association is not the within-trial modifier effect — so model the within- and across-trial interactions separately (Phillippo IPD-NMA). - RWE single-arm or external-control evidence: increasingly, only one comparator has RCT evidence and the new agent has a single-arm trial plus an RWE external control, or a real-world comparative effect is brought into a network of RCTs. Failure mode: an RWE-derived contrast carries confounding that randomization removed in the RCT arms — folding it into an NMA propagates that confounding network-wide. Anchor on a common comparator, down-weight or sensitivity-test the RWE node, and prefer population-adjustment (MAIC/STC) when the RWE and RCT populations differ on effect modifiers. FDA and EMA accept population-adjusted indirect comparisons in this situation but expect explicit, defended assumptions; NICE DSU TSD-18 and CADTH give detailed conduct standards. - Linked claims/registry-derived effects: RWE contrasts computed from claims (active-comparator new-user designs) or registries can populate a node, but the time-zero, washout, and outcome definitions must match the RCT estimand they will be combined with; otherwise the network mixes intention-to-treat RCT effects with as-treated RWE effects — a quiet estimand mismatch.

Worked example (HTA-style, claims/RCT blend)

Question: rank four biologics (A, B, C, D) for the PASI-75 response in moderate-to-severe plaque psoriasis to populate a cost-effectiveness model. Direct RCT evidence exists only for A vs placebo (P), B vs P, C vs P, and one head-to-head A vs B trial — so the network is connected through P, with a single closed loop A–B–P. (1) Node definition: each biologic at its licensed maintenance dose is one node; placebo is the reference; PASI-75 (binary) at week 12 is the common outcome. (2) Transitivity check: tabulate baseline PASI, prior-biologic exposure, and weight across the four trial sets; if the C-vs-P trial enrolled markedly more biologic-experienced patients, flag a transitivity threat and pre-specify a meta-regression on prior exposure. (3) Effect measure: log odds ratio via a binomial-logit GLM, random effects with a common τ² across comparisons. (4) Multi-arm handling: none here, but the A–B–P loop lets us node-split the A-vs-B contrast — compare the direct A-vs-B trial effect with the indirect (A-vs-P minus B-vs-P) effect and test for inconsistency. (5) Synthesis: fit the model; read the league table for all six pairwise ORs with credible intervals; if a real-world C-vs-standard-of-care comparative-effect estimate from a claims active-comparator new-user study is available, add C linked through the shared comparator only after confirming its outcome and follow-back windows match the RCT estimand, and run it as a sensitivity node. (6) Ranking: report SUCRA with the underlying ORs and a prediction interval — if B and C overlap heavily, state that their ranks are not separable. (7) Decision feed: the relative effects (not the ranks) and their full covariance enter the economic model so that parameter correlation is preserved in the PSA.

Interpreting the output

Consider the worked example: the indirect estimate for Drug A versus Drug C (via common comparator B) yields OR = exp(−0.30) ≈ 0.74, suggesting Drug A has roughly 26% lower odds of the event than Drug C.

Formal interpretation: This OR of 0.74 is an indirect estimate derived by differencing the log odds ratios from two separate trials — it is not the result of any head-to-head randomization of A against C. Its validity rests on the transitivity assumption: that the patients in the A-vs-B trial and the patients in the C-vs-B trial are sufficiently similar that treatment B is a meaningful common comparator. Transitivity is a substantive clinical judgment, not a statistical test; it is untestable from the data alone and must be evaluated by comparing baseline characteristics across trial populations. A consistency check (node-splitting) can detect whether the direct A-vs-B evidence contradicts the indirect pathway, but consistency does not prove transitivity holds. If the C-vs-B trial enrolled substantially more biologic-experienced patients than the A-vs-B trial, the 0.74 reflects a mix of populations and should not be reported as a single universal comparison.

Practical interpretation: In an HTA submission, the league-table OR of 0.74 for A vs C may support formulary preference for A, but the credible interval around that estimate — and a prediction interval if heterogeneity is present — must accompany it. Report SUCRA as a supplementary ranking aid, not as the primary decision input; treatments with overlapping credible intervals have indistinguishable ranks. Feed the full covariance matrix of pairwise contrasts into the PSA, not just the point estimates, so that parameter uncertainty is preserved when the economic model is driven by the NMA outputs.

Worked example

Scenario

Three trials have been published for a new drug class treating a chronic condition. Trial 1 compared Drug A versus Drug B (the standard of care) and found a log odds ratio of 0.70 favoring A. Trial 2 compared Drug C versus Drug B and found a log odds ratio of 0.40 favoring C. No trial ever put A and C in the same study. A health technology assessment body needs to know how A and C compare directly so the cheaper option can be selected for the formulary. Network meta-analysis derives that indirect A-vs-C estimate using B as the common comparator.

Dataset

Published trial results — one row per direct comparison. Log odds ratios are on the log scale; negative values favor the active drug over the comparator.

Steps

• Both trials used Drug B as the comparator, so B is the common anchor that connects A and C in the network.

• Write each result as a contrast versus B: log-OR(A vs B) = -0.70 and log-OR(C vs B) = -0.40.

• The indirect log-OR for A vs C equals log-OR(A vs B) minus log-OR(C vs B): (-0.70) - (-0.40) = -0.30.

• Exponentiate to recover the odds ratio on the natural scale: exp(-0.30) = 0.74.

• An odds ratio of 0.74 means Drug A has roughly 26% lower odds of the event than Drug C, based solely on the indirect chain through B.

Result

Indirect log-OR(A vs C) = (-0.70) - (-0.40) = -0.30; OR = exp(-0.30) = 0.74, favoring Drug A over Drug C — a comparison no single trial ever made directly.

Runnable example

library(netmeta)

# arm_df has one row per trial-arm: study_id, treatment, events, n
# Convert arm-level binary data to within-trial log-OR contrasts with correct
# multi-arm covariance. sm = "OR"; the binomial scale is handled by metabin internally.
p <- pairwise(treat = treatment, event = events, n = n,
              studlab = study_id, data = arm_df, sm = "OR")

# Random-effects (common tau^2) network meta-analysis, placebo as reference.
net <- netmeta(TE, seTE, treat1, treat2, studlab,
               data = p, sm = "OR",
               common = FALSE, random = TRUE, reference.group = "placebo")

print(summary(net))                 # all pairwise ORs with 95% CI + prediction intervals
netleague(net, digits = 2)          # league table for the economic model / appendix

# Inconsistency: global decomposition + per-loop net-heat; node-split on closed loops.
decomp.design(net)                  # design-by-treatment (Q) inconsistency decomposition
netheat(net, random = TRUE)         # net-heat hot spots flag inconsistent designs
print(netsplit(net))                # direct vs indirect per contrast (testable loops only)

# Ranking: report P-scores WITH the underlying effects; never a rank alone.
netrank(net, small.values = "bad")  # P-score (frequentist SUCRA analogue)

import numpy as np
import pandas as pd
import pymc as pm

# arm_df: study_id, treatment, events, n  (long, one row per arm)
studies = arm_df["study_id"].astype("category")
treats  = arm_df["treatment"].astype("category")
ref     = "placebo"                       # network reference treatment
t_codes = treats.cat.categories
ref_ix  = list(t_codes).index(ref)

s_idx = studies.cat.codes.to_numpy()
t_idx = treats.cat.codes.to_numpy()
n_studies = studies.cat.categories.size
n_treat   = t_codes.size
events    = arm_df["events"].to_numpy()
n         = arm_df["n"].to_numpy()

# Lu-Ades contrast parameterization needs each study's OWN baseline treatment (the
# first arm listed per study) and a mask flagging which arms are that baseline arm.
base_t_by_study = (arm_df.assign(_t=t_idx, _s=s_idx)
                         .groupby("_s")["_t"].first()
                         .reindex(range(n_studies)).to_numpy())
base_t_idx = base_t_by_study[s_idx]           # baseline treatment code for each arm's study
nb_pos     = np.where(t_idx != base_t_idx)[0]  # row positions of non-baseline arms

with pm.Model() as nma:
    mu    = pm.Normal("mu", 0.0, 10.0, shape=n_studies)        # study baselines
    tau   = pm.HalfNormal("tau", 1.0)                          # common heterogeneity SD
    # Basic parameters d (vs reference); reference effect fixed at 0 via masking.
    d_raw = pm.Normal("d_raw", 0.0, 10.0, shape=n_treat)
    d     = pm.Deterministic("d", pm.math.set_subtensor(d_raw[ref_ix], 0.0))
    # Random treatment effect per NON-baseline arm: mean = d[t] - d[study_baseline], SD = tau.
    # The study-specific baseline arm contributes exactly 0 (no heterogeneity added to baselines).
    delta_nb = pm.Normal("delta_nb",
                         mu=(d[t_idx] - d[base_t_idx])[nb_pos],
                         sigma=tau, shape=len(nb_pos))
    delta = pm.math.set_subtensor(
        pm.math.zeros(len(events))[nb_pos], delta_nb)
    logit = mu[s_idx] + delta
    pm.Binomial("y", n=n, logit_p=logit, observed=events)
    idata = pm.sample(2000, tune=2000, target_accept=0.95, chains=4)

# Pairwise log-OR X vs Y = d[X] - d[Y]; exponentiate for league table.
post = idata.posterior["d"].stack(s=("chain", "draw")).values   # (n_treat, n_draws)
league_or = {(a, b): float(np.exp(np.mean(post[i] - post[j])))
             for i, a in enumerate(t_codes) for j, b in enumerate(t_codes) if i != j}
# SUCRA from the posterior ranks (report WITH the contrasts and credible intervals).
ranks = (-post).argsort(axis=0).argsort(axis=0) + 1
sucra = {t: float((n_treat - ranks[i].mean()) / (n_treat - 1)) for i, t in enumerate(t_codes)}