MAIC and STC: Population-Adjusted Indirect Comparisons

In plain language

Sometimes a decision needs to compare drug B with drug C, but no trial ever tested them against each other - one company has a trial of B versus an older drug A, and the only published evidence on C is a separate trial of C versus A. You could subtract the two results, but that only works if the two trials enrolled similar patients, which they rarely do. MAIC and STC fix this using the patient-by-patient data from the B trial: MAIC reweights those patients so the B trial "looks like" the C trial's patient mix, and STC fits a prediction model to do the same job. If both trials share the older drug A as a common yardstick (an "anchored" comparison), you only need to balance the things that change the size of the treatment effect; without that shared yardstick you must balance everything and hope nothing is missing - a much shakier bet. The big catch: reweighting can quietly shrink your usable sample to a handful of patients, so a confident-looking comparison can actually rest on very little.

Health-technology decisions constantly need a comparison that no randomized trial ever ran: drug B versus drug C, when the manufacturer holds a trial of B vs A (with patient-level data) and the only evidence on C is a published C vs A trial reporting group means, not individuals. A naive cross-trial comparison ("B beat A by 20 points, C beat A by 12, so B beats C by 8") is the Bucher anchored indirect comparison, and it is valid only if the two trials enrolled exchangeable populations. They almost never do: the B vs A trial may be younger, less pre-treated, or healthier than the C vs A trial, and any factor that modifies the treatment effect then biases the indirect estimate. Population-adjusted indirect comparisons (PAICs) fix the population mismatch using the one asset a manufacturer has but the literature does not - the individual patient data from its own trial.

The two methods

MAIC (Signorovitch 2010) reweights each IPD patient so the weighted means of the selected baseline variables in the IPD trial equal the published means from the aggregate trial - a survey-style reweighting, with weights estimated by a method-of-moments / logistic-propensity trick. The reweighted IPD trial is then re-analyzed and compared with the aggregate trial. STC (simulated treatment comparison) instead fits an outcome regression in the IPD, including the effect modifiers, and uses it to predict what the IPD treatment's outcome would have been in the aggregate trial's population (by plugging in the aggregate trial's covariate means). MAIC is a weighting estimator; STC is a regression (outcome-model) estimator - the same anchored/unanchored logic applies to both. NICE DSU Technical Support Document 18 (Phillippo 2018) is the methodological reference that HTA bodies cite, and it draws the sharp line between anchored and unanchored comparisons.

Anchored vs unanchored - the assumption that decides everything

In an anchored PAIC both trials share a common comparator A, so the indirect contrast is built on the within-trial relative effects (B-vs-A and C-vs-A). Randomization inside each trial handles prognostic factors; you only need to balance effect modifiers (variables that change the size of the treatment effect) to make the two relative effects comparable. This is a strong but sometimes plausible assumption. In an unanchored PAIC there is no common comparator (e.g., a single-arm trial of B versus a single-arm or external source for C): the contrast is between absolute outcomes, randomization protects nothing, and you must adjust for every prognostic factor AND every effect modifier - and assume there are no unmeasured ones. That is the same heroic conditional-exchangeability assumption an observational external-control analysis makes, with none of the within-trial randomization to lean on. TSD 18 is explicit: unanchored comparisons are acceptable only when an anchored one is impossible, and their results should be treated with great caution.

Pros, cons, and trade-offs

(specific and comparative). - vs network-meta-analysis / Bucher (the standard anchored synthesis): Standard NMA and Bucher assume the trials are similar enough that the common-comparator anchor cancels all cross-trial differences. PAICs relax that by explicitly rebalancing the IPD trial to the aggregate trial's population, so they are the right tool when there are few trials (often just two), no closed loop, and meaningful effect-modifier imbalance. The cost: PAICs need IPD on at least one side (NMA does not), they correct only observed, reported modifiers, and they collapse the usable sample size (see ESS below). Prefer standard NMA/Bucher when a connected network of similar trials exists and imbalance is minor; prefer a PAIC when the network is a single disconnected pair and a named effect modifier differs across the two trials. - MAIC vs STC: MAIC makes no assumption about the form of the outcome model but throws away information by weighting (the effective sample size can crater, and weights are unstable when populations barely overlap). STC keeps all patients and is more efficient, but it bets on the regression being correctly specified, and - in its common, naive implementation - it can only target the aggregate trial's mean covariates, which is exact only for collapsible, linear-predictor outcomes; for nonlinear links (logistic, Cox) plugging in mean covariates is an approximation. Prefer MAIC when you distrust the outcome model and have decent overlap; prefer STC when overlap is poor (weights would explode) and you trust a parsimonious, correctly specified model. - Scale dependence (a trap both share): the adjustment is done on a particular effect scale (log-odds, log-HR, risk difference). An anchored PAIC assumes the conditional effect is constant across the part of the covariate space being matched, on that scale. Because relative effects are non-collapsible on the odds/hazard scale, a comparison that looks unbiased on one scale need not be on another, and the reported result can depend on whether you adjusted on the log-odds or the probability scale. State the scale; do not switch it silently.

When to use

A decision needs B vs C; the evidence is a disconnected pair of trials (or single arms) with no head-to-head and no usable common network; you hold IPD on at least one side; and there is a named, measured effect modifier (and, for unanchored, every prognostic factor) that differs between the two trial populations and is reported in the aggregate publication. This is the bread-and-butter situation in HTA submissions (NICE, CADTH, and the EU Joint Clinical Assessment), where a manufacturer's pivotal trial must be compared with a competitor's trial to fill a PICO the regulator or payer specifies, and where the company's IPD is the lever that lets it rebalance to the comparator trial's population.

When NOT to use - and when it is actively misleading

- Effective sample size collapse. MAIC weights can concentrate almost all of the analysis on a handful of patients. The effective sample size (ESS = (sum of weights)^2 / sum of squared weights) measures how many independent patients the weighted analysis really behaves like. When the IPD trial barely overlaps the aggregate population, ESS can fall to a small fraction of the nominal n, confidence intervals widen, and the estimate is driven by a few extreme-weight patients - a fragile result dressed up as a trial comparison. Always report ESS and a weight histogram; an ESS that is a tiny fraction of n is a red flag, not a footnote. - Unanchored when anchored was possible. Dropping the anchor to get a tidier number throws away the only randomization-based protection you had and silently swaps in a no-unmeasured-confounding assumption. Do not run an unanchored comparison if a common comparator exists; if it genuinely does not, label the result as hypothesis-generating and stress-test every prognostic assumption. - Adjusting for the wrong, or incomplete, variable set. PAICs balance only what is measured in the IPD and reported in the aggregate publication. A modifier that is unreported in the competitor's paper cannot be matched; an unanchored analysis missing one prognostic factor is biased with no diagnostic to reveal it. If the aggregate publication does not report a known effect modifier's distribution, the comparison cannot be trusted, however sophisticated the weighting. - Extrapolating beyond overlap. Reweighting cannot conjure patients the IPD trial never enrolled. If the aggregate population sits largely outside the IPD trial's covariate range, MAIC weights are enormous and unstable and STC is extrapolating off the data - both are unreliable, and a more honest answer is "the trials are too different to compare."

Data-source and evidence-source depth

Classically the IPD comes from a manufacturer's randomized trial (the cleanest substrate, with adjudicated outcomes and protocol-collected baseline covariates), and the aggregate side from a published competitor trial. Increasingly, real-world data feed both sides of a PAIC: a single-arm trial may be compared with an external control drawn from claims, EHR, or a disease registry, which turns an unanchored MAIC into an external-control study with all of that design's confounding risks. The effect modifiers themselves are often the binding constraint - age, prior lines of therapy, biomarker status, performance status - and whether each is measured in the IPD and reported in the aggregate source determines whether the comparison is even attemptable. When the aggregate side is RWE rather than a trial, the analyst must also reconcile real-world outcome definitions and follow-up with the trial's protocol endpoints before any matching is meaningful.

Interpreting the output

Consider the worked example: after reweighting B-trial patients to match the competitor trial's 60% prior-biologic rate, the anchored MAIC yields a risk difference of 0.08 (8 percentage points) in favor of B over C. The effective sample size fell to 2.5 of the original 5 patients.

Formal interpretation: The 0.08 estimate is a population-adjusted indirect comparison, anchored through the shared comparator A. It targets the treatment effect of B versus C in the population of the competitor's trial — not in the population of the B trial as originally enrolled. The ESS deflation from 5 to 2.5 reflects how concentrated the reweighting became: a single up-weighted patient dominates the estimate, and the resulting confidence interval is wide. In an anchored MAIC, the indirect comparison is only as credible as the assumption that A performs identically in both trial populations after reweighting — an untestable assumption. In an unanchored MAIC (no shared comparator), the additional assumption of absolute outcome exchangeability makes the result even more fragile and should be labeled explicitly as such in any HTA submission.

Practical interpretation: Report the ESS alongside the point estimate as a transparency requirement, not as an afterthought. An ESS below 40% of the original sample is a recognized red flag in NICE technical guidance. The 0.08 risk difference and its interval should be accompanied by a weight distribution plot and a sensitivity analysis varying the set of effect modifiers included. If the aggregate data source is RWE rather than a published trial, reconcile outcome definitions and follow-up windows before matching — a mismatch here invalidates the comparison regardless of how well the weights balance the covariates.

Worked example

Scenario

A manufacturer must show how its drug B compares with a competitor's drug C, but no head-to-head trial exists. It holds patient-level data from its own trial of B versus the older drug A, and the only evidence on C is a published trial of C versus A. The two trials share comparator A, so an anchored MAIC is possible. One baseline factor - whether a patient had a prior biologic - is known to change the treatment effect and differs between the trials: only 1 of the 5 patients in the B trial had a prior biologic, while the published C trial reports 60% did. We reweight the B-trial patients so their prior-biologic rate matches 60%, check how much usable sample survives, and form the anchored B-versus-C comparison.

Dataset

The five patient-level rows from the B-versus-A trial (the IPD side), plus the one summary number the competitor's paper reports.

Steps

• The B trial's prior-biologic rate is 1 of 5 patients = 1 / 5 = 0.20, but the published C trial reports 0.60 - a real population mismatch in a known effect modifier.

• Reweight so the weighted prior-biologic rate hits 0.60. Method-of-moments weights give the single prior-biologic patient a weight of 6 and each of the 4 others a weight of 1; check the weighted rate = 6 / (6 + 4) = 0.60.

• Sum of the weights = 6 + 4 = 10. Sum of the squared weights = 6 * 6 + 4 = 40 (one patient contributes 36, four contribute 1 each).

• Effective sample size = (sum of weights) squared over sum of squared weights = 10 * 10 / 40 = 2.5, so the 5 nominal patients behave like only 2.5 independent ones - a 50% collapse, the key warning sign.

• Recompute the B-versus-A effect on the reweighted population (risk difference 0.20) and take the published C-versus-A effect (risk difference 0.12); the anchored indirect estimate is B vs C = 0.20 - 0.12 = 0.08.

Result

After matching the prior-biologic rate to the competitor trial's 0.60, the anchored MAIC estimates that B improves response by 0.08 (8 percentage points) more than C. But the effective sample size fell to 2.5 of 5 patients, so the interval around 0.08 is wide and the result leans heavily on a single up-weighted patient - report the ESS, not just the point estimate.

Runnable example

import numpy as np
import pandas as pd
from scipy.optimize import minimize

def maic_anchored(ipd: pd.DataFrame, mod_cols, agg_means: dict,
                  agg_effect_CA: float, outcome="outcome", arm="arm"):
    # 1) Center each effect modifier on its aggregate-trial target mean.
    #    Balance is achieved when the WEIGHTED sum of centered covariates is zero.
    X = ipd[mod_cols].to_numpy(dtype=float)
    Xc = X - np.array([agg_means[c] for c in mod_cols])

    # 2) Method-of-moments weights w_i = exp(Xc_i . a). The objective
    #    Q(a) = sum(exp(Xc . a)) is convex; its gradient is Xc' w, which is
    #    exactly the weighted covariate imbalance, so the minimizer balances the means.
    def Q(a):  return np.sum(np.exp(Xc @ a))
    def dQ(a): return Xc.T @ np.exp(Xc @ a)
    a0 = np.zeros(Xc.shape[1])
    a_hat = minimize(Q, a0, jac=dQ, method="BFGS").x
    w = np.exp(Xc @ a_hat)

    # 3) Effective sample size = (sum w)^2 / sum(w^2): how many independent
    #    patients the weighted analysis behaves like. A small ESS is a red flag.
    ess = float(w.sum() ** 2 / np.sum(w ** 2))

    # 4) Weighted within-trial B-vs-A effect (risk difference) on the matched population.
    d = ipd.assign(_w=w)
    def wmean(g): return np.average(g[outcome], weights=g["_w"])
    rB = wmean(d[d[arm] == "B"]); rA = wmean(d[d[arm] == "A"])
    effect_BA = rB - rA

    # 5) Anchored (Bucher) indirect comparison on the common comparator A.
    effect_BC = effect_BA - agg_effect_CA
    return {"ess": round(ess, 2), "weighted_effect_BA": round(effect_BA, 4),
            "indirect_effect_BC": round(effect_BC, 4),
            "weight_ratio_max": round(float(w.max() / w.min()), 2)}

maic_anchored <- function(ipd, mod_cols, agg_means, agg_effect_CA,
                          outcome = "outcome", arm = "arm") {
  # 1) Center modifiers on the aggregate-trial target means.
  X  <- as.matrix(ipd[, mod_cols, drop = FALSE])
  Xc <- sweep(X, 2, agg_means[mod_cols], "-")

  # 2) Method-of-moments weights via convex objective Q(a) = sum(exp(Xc %*% a)).
  Q  <- function(a) sum(exp(Xc %*% a))
  dQ <- function(a) as.vector(t(Xc) %*% exp(Xc %*% a))
  a_hat <- optim(rep(0, ncol(Xc)), Q, gr = dQ, method = "BFGS")$par
  w <- as.vector(exp(Xc %*% a_hat))

  # 3) Effective sample size = (sum w)^2 / sum(w^2).
  ess <- sum(w)^2 / sum(w^2)

  # 4) Weighted within-trial B-vs-A risk difference on the matched population.
  wmean <- function(idx) weighted.mean(ipd[[outcome]][idx], w[idx])
  rB <- wmean(ipd[[arm]] == "B"); rA <- wmean(ipd[[arm]] == "A")
  effect_BA <- rB - rA

  # 5) Anchored (Bucher) indirect comparison through common comparator A.
  effect_BC <- effect_BA - agg_effect_CA
  list(ess = round(ess, 2),
       weighted_effect_BA = round(effect_BA, 4),
       indirect_effect_BC = round(effect_BC, 4),
       weight_ratio_max = round(max(w) / min(w), 2))
}