Multi-Criteria Decision Analysis (MCDA)

In plain language

Multi-criteria decision analysis (MCDA) is a way to compare treatment options when several things matter at once - how well a drug works, how safe it is, how badly patients need something new. A committee first agrees on the list of criteria, scores each option on each criterion from 0 to 100, and assigns weights that say how much each criterion matters; each option's weighted scores are then added up into one total for comparison. The total makes the committee's trade-offs visible and checkable, but it is only as good as the weights - different people, methods, or framings can produce different weights and sometimes a different winner, which is why the ranking is meant to structure the discussion, not replace it.

Most health decisions are multi-attribute whether we admit it or not: an HTA committee weighing a drug is trading off survival gain against toxicity against unmet need against budget; a payer prioritizing a formulary is doing the same across products; a regulator's benefit-risk call balances effect sizes against harms. MCDA makes that implicit weighing explicit. The canonical process (ISPOR MCDA Emerging Good Practices Task Force, Thokala et al. 2016; Marsh et al. 2016) is: (1) structure the problem - define the decision, the alternatives, and a criteria set that is complete, non-redundant, and preferentially independent; (2) measure performance - build a performance matrix of each alternative on each criterion from trials, RWE, and elicitation; (3) score - convert natural-unit performance to a common 0-100 partial value scale against declared worst and best anchors (linear or elicited value functions); (4) weight - elicit how much a swing from worst to best on each criterion matters relative to the others; (5) aggregate - usually the additive value model V(a) = sum over criteria of w_k x s_k(a), with weights normalized to sum to 1; (6) test - sensitivity analysis on weights and scores; (7) deliberate - the numbers structure the discussion, they do not end it.

Value measurement methods

Swing weighting asks the committee to imagine the worst hypothetical alternative and rank-then-rate which single criterion swing (worst to best) they would fix first; the top swing gets 100 points, others are rated relative to it, and points are normalized to weights. It is the method most consistent with the additive model because weights are anchored to the actual criterion ranges - a weight only means anything relative to the swing it covers. AHP (Analytic Hierarchy Process) derives weights from pairwise comparisons on a 1-9 verbal scale via the principal eigenvector, with a consistency ratio to flag incoherent judgments; it is easy to field but its verbal scale and rank-reversal behavior are theoretically contested. DCE-derived weights estimate them from choice experiments over attribute profiles, importing preference-study machinery (and its sample, framing, and attribute-range dependence) into the weight set. Outranking methods (ELECTRE, PROMETHEE) avoid full aggregation by pairwise-comparing alternatives with concordance thresholds - useful when trade-offs are contested, but harder to explain to a deliberative committee.

The additive model's independence assumptions are the load-bearing wall

A weighted sum is only valid when criteria are mutually preferentially independent: how much you value a swing on safety must not depend on the level of efficacy. When criteria interact (a toxicity matters more when survival gain is small), the additive form misstates value and you need multiplicative or other non-additive forms - or a re-structured criteria set. Double counting is the everyday violation: putting "QALY gain" and "quality of life" and "severity" in one criteria set counts the same value twice; cost criteria alongside health criteria quietly re-derive a cost-effectiveness threshold the committee never agreed to.

Pros, cons, and trade-offs

(specific and comparative). - vs cost-utility analysis (CUA): CUA collapses value to one metric (the QALY) and one decision rule (the ICER vs a threshold), which buys comparability across appraisals and decades of methods guidance - but it cannot natively carry unmet need, severity, equity, or innovation except as ad hoc modifiers. MCDA carries them explicitly with committee-owned weights, at the price of losing the QALY's interpersonal comparability and opportunity-cost logic: an MCDA total value of 72 has no exchange rate against the health forgone elsewhere in the budget. Prefer CUA for reimbursement decisions inside a budget-constrained system with an established threshold; prefer MCDA when the decision explicitly trades off criteria the QALY cannot hold, or where no threshold logic exists (portfolio triage, early pipeline, orphan/severity frameworks). - vs deliberation alone (unaided committee judgment): Unaided deliberation is flexible and cheap but opaque - weights live in members' heads, anchoring and loudest-voice effects go unmeasured, and consistency across meetings is unauditable. MCDA forces the value judgments into the open where they can be challenged and reused. The cost is real: elicitation burden, false precision risk, and the temptation to treat the score as the decision. - vs preference studies (DCE/conjoint) as the weight source: Committee swing weighting is fast and produces weights owned by the actual decision makers, but from a handful of people. DCE-derived weights bring a defensible sample (patients, public) and statistical machinery, but the weights inherit the experiment's attribute ranges and framing, and the committee may not feel bound by preferences it did not express.

When to use

Deliberative HTA appraisal where severity, unmet need, or equity must enter transparently rather than as unexplained discretion; portfolio and formulary prioritization across many candidates with the same criteria set; structured quantitative benefit-risk for regulatory or pharmacovigilance decisions (weighted benefits vs harms with explicit trade-offs); resource allocation where stakeholders disagree and the disagreement should be located in weights, not buried; early-stage value frameworks before enough data exist for full economic modeling.

When NOT to use - and when it is actively misleading

- Do not use MCDA as a substitute for economic evaluation in a budget-constrained reimbursement decision. Total value scores carry no opportunity-cost information; ranking by MCDA score and funding down the list can displace more health than it buys. If cost enters at all, keep it outside the value score (value-for-money displays) rather than as a weighted criterion. - Double counting. Overlapping criteria (efficacy + QoL + severity that is itself defined by efficacy shortfall) silently multiply one attribute's weight. Criteria sets must be tested for redundancy before any weight is elicited. - Weight elicitation fragility. Weights move with the elicitation method (swing vs AHP vs DCE on the same problem yield different weights), with attribute ranges (halve the efficacy range and its swing weight should roughly halve - committees routinely fail this range sensitivity check), with framing, and with who is in the room. Reporting a single weighted total without weight sensitivity analysis is misleading precision. - Independence violations. If criterion values interact, the additive sum is the wrong functional form - check preferential independence explicitly during problem structuring, not after the ranking is computed. - Score laundering. When the performance matrix cells come from weak or heterogeneous evidence (a registry rate next to an RCT effect next to an expert guess), the tidy 0-100 scores hide the uncertainty gradient; carry evidence uncertainty into the sensitivity analysis or display it alongside the scores.

Interpreting the output

Consider the worked example: Drug A scores 72 and Drug B scores 67 under the committee's weights (efficacy 0.5, safety 0.3, unmet need 0.2), placing Drug A first by 5 points.

Formal interpretation: The weighted total of 72 is a composite index constructed by multiplying each criterion's 0–100 rescaled performance score by a normalized weight and summing. The 5-point gap is only as meaningful as the weights that produced it: weight elicitation is known to be sensitive to the method used (swing weighting, AHP, DCE), the attribute ranges presented (halving the efficacy anchor range should roughly halve efficacy's swing weight — committees routinely fail this range-sensitivity check), and the composition of the eliciting group. The 0–100 rescaling assumes linearity within each criterion's range; if the committee's preferences are non-linear (e.g., diminishing marginal value of additional OS gain beyond 8 months), the additive model mis-ranks alternatives. Preferential independence — required for the additive model to be valid — must be checked during problem structuring, not inferred from a clean-looking output.

Practical interpretation: Report the weighted total alongside the underlying performance matrix and the weights, not as a standalone score. Show the weight-sensitivity threshold at which the ranking flips — in this case, if the efficacy weight falls below the flip point, Drug B wins — so deliberation focuses on whether the committee is confident enough in the efficacy weight to sustain the Drug A ranking. Do not use the total score for cost-effectiveness inference: MCDA scores carry no opportunity-cost information and cannot substitute for an incremental cost- effectiveness ratio.

Worked example

Scenario

An HTA committee compares two drugs for the same disease using three criteria - efficacy (overall-survival gain in months), safety (serious adverse events per 100 patients), and unmet need addressed (a committee score from 0 to 100). Anchors were agreed in advance: survival gain runs from 0 (worst) to 10 months (best), the adverse-event rate from 20 per 100 (worst) to 0 (best), and unmet need is already on a 0-100 scale. In the swing-weighting session the committee put the efficacy swing first at 100 points, the safety swing at 60, and the unmet-need swing at 40. We rescale each drug's performance to 0-100 scores, normalize the weights, and add up the weighted scores.

Dataset

The performance matrix the committee sees - each drug's measured performance per criterion in natural units (swing points elicited separately - efficacy 100, safety 60, unmet need 40).

Steps

• Normalize the swing points to weights that sum to 1. Total points = 100 + 60 + 40 = 200, so efficacy weight = 100/200 = 0.5, safety weight = 60/200 = 0.3, unmet-need weight = 40/200 = 0.2.

• Rescale efficacy to a 0-100 score between the anchors (0 worst, 10 best). Drug A = (8-0)/(10-0)100 = 80; Drug B = (6-0)/(10-0)100 = 60.

• Rescale safety the same way, remembering lower is better (20 worst, 0 best). Drug A = (20-8)/(20-0)100 = 60; Drug B = (20-2)/(20-0)100 = 90.

• Unmet need is already on the 0-100 scale, so Drug A scores 70 and Drug B scores 50.

• Add up weight times score for Drug A. Total value = 0.580 + 0.360 + 0.2*70 = 40 + 18 + 14 = 72.

• Add up weight times score for Drug B. Total value = 0.560 + 0.390 + 0.2*50 = 30 + 27 + 10 = 67.

• Compare and stress-test. Drug A leads by 72 - 67 = 5 points on the back of efficacy; if the committee dropped the efficacy weight to 0.2 (with safety 0.48 and unmet need 0.32), Drug B would win - so report that the ranking turns on the efficacy weight.

Result

Drug A total value = 72, Drug B total value = 67 - Drug A ranks first by 5 points under the committee's weights (0.5 efficacy, 0.3 safety, 0.2 unmet need), and sensitivity analysis shows the ranking flips if the efficacy weight falls far enough, so the deliberation should focus on how firmly the committee holds that weight.

Timeline Spec

Title

One HTA committee MCDA cycle - scoping, weighting, scoring, deliberation

Window

Start

2025-09-01

End

2025-10-24

Label

Eight-week MCDA cycle from scoping workshop to decision

Events

• Label

  Scoping workshop

  Start

  2025-09-01

  Length Days

  14

  Quantity

  3 criteria + anchors agreed

• Label

  Swing-weight elicitation

  Start

  2025-09-15

  Length Days

  14

  Quantity

  points 100 / 60 / 40

• Label

  Performance matrix + scoring

  Start

  2025-09-29

  Length Days

  21

  Quantity

  scores A: 80/60/70, B: 60/90/50

• Label

  Deliberation + sensitivity

  Start

  2025-10-20

  Length Days

  5

  Quantity

  totals A=72, B=67

Spans

• Kind

  exposed

  Start

  2025-09-01

  End

  2025-10-19

  Label

  Model build: criteria, weights, evidence scoring

• Kind

  followup

  Start

  2025-10-20

  End

  2025-10-24

  Label

  Decision week: weighted totals structure deliberation

Result

Label

Drug A total value 72 vs Drug B 67 - A ranked first under committee weights

Value

72

Runnable example

import pandas as pd

# Performance matrix in NATURAL units (3 criteria x 2 alternatives).
perf = pd.DataFrame(
    {"os_gain_months": [8.0, 6.0],      # efficacy: overall-survival gain vs standard of care
     "sae_rate_per100": [8.0, 2.0],     # safety: serious adverse events per 100 patients (lower = better)
     "unmet_need_score": [70.0, 50.0]}, # committee-scored unmet need addressed, already on 0-100
    index=["Drug A", "Drug B"])

# Declared anchors: worst and best PLAUSIBLE levels per criterion (set during problem structuring,
# BEFORE weights are elicited - swing weights are only meaningful relative to these ranges).
anchors = {                      # (worst, best)
    "os_gain_months":   (0.0, 10.0),
    "sae_rate_per100":  (20.0, 0.0),   # harm: worst is the HIGH rate
    "unmet_need_score": (0.0, 100.0),
}

# Raw swing-weight points: top-ranked swing = 100, others rated relative to it.
swing_points = {"os_gain_months": 100.0, "sae_rate_per100": 60.0, "unmet_need_score": 40.0}

def partial_value(x: float, worst: float, best: float) -> float:
    """Linear 0-100 partial value between anchors; direction-aware via the anchor order."""
    return 100.0 * (x - worst) / (best - worst)

def additive_mcda(perf: pd.DataFrame, anchors: dict, swing_points: dict) -> pd.DataFrame:
    total_pts = sum(swing_points.values())
    weights = {k: v / total_pts for k, v in swing_points.items()}   # normalize to sum to 1
    scores = perf.apply(lambda col: partial_value(col, *anchors[col.name]), axis=0)
    contrib = scores * pd.Series(weights)            # weighted contribution per criterion
    out = contrib.add_suffix("_wtd")
    out["total_value"] = contrib.sum(axis=1)
    out["rank"] = out["total_value"].rank(ascending=False).astype(int)
    return out.round(2)

result = additive_mcda(perf, anchors, swing_points)
print(result)
# Drug A: 0.5*80 + 0.3*60 + 0.2*70 = 72.0 (rank 1); Drug B: 0.5*60 + 0.3*90 + 0.2*50 = 67.0 (rank 2)

# Minimal weight sensitivity: at what efficacy weight do the alternatives tie?
for w_eff in (0.50, 0.40, 0.30, 0.20):
    rest = 1.0 - w_eff
    w = {"os_gain_months": w_eff, "sae_rate_per100": rest * 0.6, "unmet_need_score": rest * 0.4}
    scores = perf.apply(lambda col: partial_value(col, *anchors[col.name]), axis=0)
    tot = (scores * pd.Series(w)).sum(axis=1)
    print(f"w_efficacy={w_eff:.2f}: A={tot['Drug A']:.1f}  B={tot['Drug B']:.1f}")

# Performance matrix in NATURAL units (rows = alternatives, cols = criteria).
perf <- rbind(
  `Drug A` = c(os_gain_months = 8, sae_rate_per100 = 8, unmet_need_score = 70),
  `Drug B` = c(os_gain_months = 6, sae_rate_per100 = 2, unmet_need_score = 50)
)

# Declared worst/best anchors per criterion (direction-aware: for the harm, worst is the HIGH rate).
anchors <- list(
  os_gain_months   = c(worst = 0,  best = 10),
  sae_rate_per100  = c(worst = 20, best = 0),
  unmet_need_score = c(worst = 0,  best = 100)
)

# Raw swing-weight points (top swing = 100), normalized to weights summing to 1.
swing_points <- c(os_gain_months = 100, sae_rate_per100 = 60, unmet_need_score = 40)
weights <- swing_points / sum(swing_points)   # 0.5, 0.3, 0.2

partial_value <- function(x, worst, best) 100 * (x - worst) / (best - worst)

# 0-100 partial value scores, then weighted contributions and total value.
scores  <- sapply(colnames(perf), function(k)
             partial_value(perf[, k], anchors[[k]]["worst"], anchors[[k]]["best"]))
contrib <- sweep(scores, 2, weights, `*`)
total   <- rowSums(contrib)

out <- data.frame(round(contrib, 2), total_value = round(total, 2),
                  rank = rank(-total))
print(out)
# Drug A: 0.5*80 + 0.3*60 + 0.2*70 = 72 (rank 1); Drug B: 0.5*60 + 0.3*90 + 0.2*50 = 67 (rank 2)

# Weight sensitivity: sweep the efficacy weight, split the remainder 60/40 safety:unmet need.
for (w_eff in c(0.5, 0.4, 0.3, 0.2)) {
  w   <- c(w_eff, (1 - w_eff) * 0.6, (1 - w_eff) * 0.4)
  tot <- scores %*% w
  cat(sprintf("w_efficacy=%.2f: A=%.1f  B=%.1f\n", w_eff, tot[1], tot[2]))
}