ISPOR Good Practices for Indirect Treatment Comparisons & Network Meta-Analysis

What it is

— The ISPOR Indirect Treatment Comparisons (ITC) Good Research Practices Task Force reports, issued by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR), are the field's reference standard for indirect and mixed treatment comparisons and network meta-analysis (NMA). The guidance comes in two complementary parts. Part 1 (Jansen et al., 2011) explains how to interpret ITC/NMA: the assumptions (similarity, homogeneity, consistency), the difference between anchored and unanchored comparisons, fixed- vs random-effects models, and how to read relative-effect and ranking outputs. Part 2 (Hoaglin et al., 2011) is the conducting and reporting companion: a checklist of items a credible ITC/NMA must document, from the systematic-review base and network diagram through statistical model, software, and presentation of results. The later ISPOR-AMCP-NPC ITC/NMA Study Questionnaire (Jansen et al., 2014) turns the principles into a structured relevance-and-credibility appraisal instrument that payers and HTA bodies use to decide whether a submitted ITC/NMA can be trusted. Together these are the closest thing the indirect-comparison literature has to STROBE/PRISMA-grade governance, and NICE, CADTH, and other HTA agencies cite them directly.

When to use

— Reach for this guidance whenever a decision requires comparing treatments that have not been studied head-to-head, but are linked through a connected network of randomized trials sharing common comparators. Typical contexts: an HTA/payer dossier (NICE, CADTH, ICER, G-BA) positioning a new drug against comparators it was never trialled against; a regulatory submission where an anchored indirect comparison supplements direct evidence; or a peer-reviewed evidence synthesis going beyond pairwise meta-analysis. Decision rule for which tool applies: (1) if a connected network of RCTs with a shared common comparator exists, an anchored ITC/NMA under ISPOR Good Practices is appropriate and preserves within-trial randomization. (2) If the network is disconnected, or the comparison rests on single-arm/observational data with imbalanced effect modifiers, the anchored ISPOR framework no longer holds and you must move to population-adjusted methods (MAIC, STC) or external-control approaches — and say so explicitly. (3) For appraising a submitted ITC/NMA rather than building one, apply the 2014 questionnaire as the credibility checklist.

What it requires

— The substantive domains the guidance enforces are specific to network evidence, not generic study design: - Systematic-review foundation — the network must be assembled from a transparent, reproducible systematic review (PRISMA-grade search, eligibility, extraction); the ITC is only as good as the trials feeding it. - Network geometry and connectedness — a presented network diagram, evidence that the network is connected, and disclosure of how multi-arm trials and closed loops are handled. - Transitivity / similarity of effect modifiers — the central assumption: trials across the network must be sufficiently similar in distribution of effect modifiers (patient characteristics, dosing, outcome definitions, follow-up) for indirect comparison to be valid; this must be assessed, not asserted. - Homogeneity within each pairwise contrast and consistency between direct and indirect evidence on every closed loop — with formal inconsistency assessment (node-splitting, loop-specific or design-by-treatment models). - Model specification — choice of fixed- vs random-effects, the likelihood/link, priors (for Bayesian fits), handling of multi-arm correlation, convergence diagnostics, and the software used, all pre-specified and reported. - Presentation of results — relative effects for all pairs with credible/confidence intervals, and ranking metrics (SUCRA, rank probabilities, P-scores) reported with explicit caveats about their fragility. - Credibility and relevance appraisal (2014 questionnaire) — whether the analysis answers the decision-maker's PICO, and whether its assumptions are defensible enough to act on.

When NOT to use — limitations and common misapplications

— These reports are good-practice / credibility guidance, not a mechanical quality score: a completed Part 2 checklist or 2014 questionnaire documents what was done, it does not certify the answer is unbiased. The dominant failure modes: (1) Running an NMA over a network that violates transitivity — pooling trials whose populations differ systematically in effect modifiers produces a precise but biased estimate; the checklist is satisfied while the inference is wrong. (2) Ignoring inconsistency — reporting a consistency model without ever testing direct-vs-indirect agreement on closed loops. (3) Over-reading rankings — presenting SUCRA/"probability best" as if it were a robust ordering when ranks are unstable and sensitive to imprecise nodes. (4) Using the anchored ISPOR framework where it does not apply — forcing an NMA through a disconnected network, or onto single-arm/external-control evidence with imbalanced effect modifiers, where population-adjusted methods (MAIC/STC) or formal external-control designs are required instead. (5) Checklist-as-theatre — appending a completed questionnaire to a dossier while the underlying systematic review is incomplete or the model is unspecified. (6) Mistaking this guidance for one that governs observational RWE design (confounding control, time-zero, phenotyping) — those belong to RECORD-PE/HARPER/STaRT-RWE, not ISPOR ITC.

How it maps to this catalog

— The implementing concepts in this repository are the evidence-synthesis and external-comparison methods, not pharmacoepi design tools: - The network geometry, transitivity, consistency, and ranking requirements are implemented by network-meta-analysis — the core analytic method this guidance governs. - The systematic-review base and pairwise foundations are implemented by meta-analysis-rct (the usual substrate for an anchored ITC) and meta-analysis-obs when observational evidence enters the synthesis; ipd-meta-analysis implements the individual-patient-data variant that strengthens effect-modifier adjustment. - The transitivity / similarity assumption — that effect-modifier distributions are exchangeable across the network — is conceptually implemented by generalizability-transportability-external-validity-rwe, which makes the exchangeability logic explicit. - The decision boundary (when the anchored ISPOR framework fails and you must adjust for population differences or lean on external comparators) maps to single-arm-external-control and rare-disease-external-controls-rwe; the trial-protocol discipline behind a credible comparator maps to target-trial-emulation.

Applied note (RWE-anchored ITC)

ITC/NMA is fundamentally trial-data-centric, but RWE increasingly feeds these networks — e.g., a real-world external-control arm or an observational study contributing a node. When that happens, the transitivity assumption becomes far harder to defend: claims/EHR/registry cohorts differ from RCT populations in effect modifiers, outcome ascertainment, and follow-up. Document the RWD source's fitness-for-use, characterize how its population differs from the trial nodes, down-weight or sensitivity-test that node, and consider population-adjusted methods before letting observational evidence drive an indirect comparison submitted to an HTA body.