Scenario Analysis and Deterministic Sensitivity Analysis (DSA)

In plain language

When a health-economic model says a treatment is good value, the obvious question is: would that conclusion survive if the inputs or assumptions were different? Deterministic sensitivity analysis answers this by changing one input at a time — say, the drug price or a quality-of-life weight — between a justified low and high value, and watching how much the answer moves; the results are usually drawn as a tornado diagram with the most influential inputs on top. Scenario analysis asks a bigger version of the same question: it re-runs the whole model under a different assumption, like a shorter time horizon, a different discount rate, or a different way of extrapolating survival, and reports the answer for each. Together they show which inputs and choices the conclusion really hinges on, and whether any reasonable alternative flips the decision. The honest caveats: moving one input at a time understates how uncertainty combines, the bars in a tornado are only as honest as the ranges chosen, and these analyses say nothing about probability — that job belongs to probabilistic sensitivity analysis.

Scenario analysis and deterministic sensitivity analysis (DSA)

are the transparency workhorses of health-economic modeling. A one-way DSA changes a single input parameter from its base-case value to a justified low and high value while holding everything else fixed, records the resulting incremental cost-effectiveness ratio (ICER) or net monetary benefit (NMB), and repeats for each influential parameter; the results are ranked by output swing and displayed as a tornado diagram. A multi-way DSA varies two or more parameters jointly (e.g., a two-way grid of drug price × treatment effect). A scenario analysis re-runs the entire model under a discrete alternative assumption set rather than a parameter range — a shorter or lifetime time horizon, 0%/3%/5% discount rates, an alternative survival-extrapolation family (Weibull vs Gompertz), a different perspective (payer vs societal), an alternative comparator, or a different source for a key input. The two address different things: DSA probes parameter influence (which numeric inputs drive the result), while scenario analysis probes structural and methodological uncertainty (which modeling choices drive the result) — the layer that probabilistic sensitivity analysis (PSA) cannot reach, because PSA only propagates distributions within a fixed structure.

Core conceptual distinctions

(1) Influence vs decision uncertainty: DSA and scenarios answer "what matters and does the decision flip?"; PSA answers "given joint parameter uncertainty, how probable is it that the intervention is cost-effective?". They are complements mandated together by ISPOR-SMDM good practice, not substitutes. (2) Parameter vs structural uncertainty: a parameter has a plausible numeric range (a utility, a cost, a relative risk) and belongs in DSA/PSA; a structural choice (horizon, extrapolation family, cycle length, competing-risk handling) has discrete alternatives and belongs in scenario analysis. Forcing structure into a parameter range (or ignoring it because the PSA looks tight) understates uncertainty. (3) Justified ranges vs arbitrary ±20%: DSA ranges should come from confidence intervals, published extremes, or pre-specified plausibility arguments; a uniform ±20% applied to everything is a convention, not evidence, and the tornado it produces reflects the analyst's range choices as much as the data. (4) Threshold crossing: the decision-relevant output of any deterministic analysis is whether the ICER crosses the willingness-to-pay threshold (or NMB crosses zero) — a threshold analysis solves explicitly for the parameter value at which the decision flips.

Pros, cons, and trade-offs

(named against the alternatives). - vs probabilistic sensitivity analysis (PSA): DSA/scenarios are transparent, cheap, and communicable — a reviewer can re-derive every number — and they expose structural uncertainty PSA ignores. But one-way DSA understates joint uncertainty (parameters move one at a time, correlations are ignored) and provides no probability statement. Run both: DSA/scenarios for influence and structure, PSA for decision uncertainty; HTA bodies (NICE, CADTH/CDA, ICER) expect all of them. - vs tipping-point / threshold analysis: a threshold analysis is the inverted one-way DSA — instead of mapping a range to outputs, it solves for the input value where the decision changes. Prefer it when the question is "how wrong would this input have to be?"; prefer standard DSA when summarizing many parameters. - One-way vs multi-way DSA: one-way is readable but ignores interactions; multi-way captures joint movement of two or three chosen parameters but becomes uninterpretable beyond that — past two or three dimensions the correct tool is PSA. - Tornado diagram vs reporting all scenario rows: the tornado is an efficient influence ranking but hides asymmetry detail and invites range-gaming (wider stated range → longer bar → "more important"). A scenario table reporting incremental costs, incremental effects, and the resulting ICER per row is the auditable form; show both.

When to use

In every cost-effectiveness, cost-utility, and budget-impact model: to identify which inputs drive the result (model debugging and value-of-information triage), to demonstrate robustness or fragility of the base-case decision to HTA reviewers, to quantify structural uncertainty (horizon, extrapolation, perspective, discounting) that PSA cannot, and to satisfy reporting standards (CHEERS 2022 requires characterizing uncertainty; ISPOR-SMDM task-force guidance requires both deterministic and probabilistic analyses). Scenario analysis is also the vehicle for mandated reference-case variations (e.g., 0%/5% discount scenarios alongside a 3% or 3.5% base case).

When NOT to use — and when it is actively misleading or dangerous

- As a substitute for PSA. A tornado diagram makes no probability statement; presenting deterministic ranges as if they characterized decision uncertainty ("the ICER ranged from X to Y, so we are confident...") is the classic misuse. Joint parameter uncertainty requires PSA. - Cherry-picked scenarios. Running many scenarios and reporting only the favorable ones — or defining scenario assumptions after seeing results — converts an uncertainty analysis into advocacy. Pre-specify the scenario set (horizon, discounting, extrapolation, perspective) before running the model. - Arbitrary or asymmetric-by-convenience ranges. ±20% on every parameter, or wide ranges only on parameters that favor the intervention, produce a tornado that misranks influence. Ranges need a stated source (CI, literature extremes, expert elicitation) reported next to the bar. - One-way DSA on correlated parameters. Varying a survival parameter while holding its correlated counterpart fixed produces internally inconsistent model states (e.g., a shape parameter incompatible with the fixed scale); correlated blocks should move together (multi-way or PSA with correlation). - Hiding the decision flip. Reporting only output ranges without flagging which scenarios cross the willingness-to-pay threshold buries the only result that matters to the decision maker; every scenario table should mark threshold crossings explicitly.

Data-source operational depth

Scenario analysis is where RWE inputs earn their keep — and where their fragility shows. In claims-informed models, standard scenarios include alternative cost sources (paid vs allowed amounts), alternative comorbidity/risk-adjustment specifications, and restricting to FFS person-time versus pooling Medicare Advantage encounters (whose paid amounts are unreliable). In EHR/registry-informed models, scenarios swap the outcome algorithm (e.g., rwPFS definition variants) or the extrapolation family fitted to registry survival. In survey/utility inputs, scenarios swap the utility tariff or mapping algorithm. Each scenario should change one named assumption, keep everything else at base case, and report the full incremental-cost / incremental-effect / ICER row so reviewers can trace exactly what moved.

Interpreting the output

The worked example shows a base-case ICER of $44,000/QALY (cost-effective at the $50,000 threshold), with the 10-year horizon scenario producing $60,000/QALY (which flips the decision) and 0% discounting producing $40,000/QALY (robust).

(1) Formal interpretation. A tornado diagram ranks each parameter by the width of the ICER swing it produces when moved from its lower to upper plausible bound, holding everything else at the base case. The widest bar identifies the single parameter whose uncertainty most drives the cost-effectiveness conclusion. In this example, the time horizon — a structural assumption rather than a parameter — produces the largest swing and actually crosses the decision threshold: at 10 years the ICER rises to $60,000, which is above the $50,000 threshold. Each scenario row should report the full set (incremental cost, incremental effect, ICER) so reviewers can verify the arithmetic and identify whether cost, effect, or both changed.

(2) Practical interpretation. When a scenario flips the cost-effectiveness verdict — as the 10-year horizon does here — it is a critical finding, not a minor sensitivity. The DSA has identified that the decision depends materially on whether the model follows patients for 5 versus 10 years, a structural choice that the PSA would not have tested (PSA samples from parameter distributions, not from alternative structural forms). This finding should be prominently disclosed, and the rationale for the base-case horizon should be explicitly defended. Scenarios that keep the ICER well below the threshold across all tested assumptions (like 0% discounting here) provide evidence of robustness.

Worked example

Scenario

A cost-utility model finds a new treatment costs more but adds quality-adjusted life-years versus standard care, and the payer threshold is $50,000 per QALY. We compute the base-case ICER, then re-run the model under three pre-specified scenarios — a shorter 10-year time horizon, an alternative (Gompertz) survival extrapolation, and 0% discounting — and check which scenarios push the ICER across the threshold.

Dataset

Incremental cost and incremental QALYs from each model run, one row per pre-specified scenario.

Steps

• Compute the base-case ICER as incremental cost divided by incremental QALYs: 22,000 / 0.50 = 44,000 per QALY — below the $50,000 threshold, so the treatment is cost-effective in the base case.

• Re-run under the 10-year horizon: 18,000 / 0.30 = 60,000 per QALY — above the threshold; truncating the horizon cuts off late QALY gains faster than late costs, and the decision flips.

• Re-run under the Gompertz extrapolation: 21,000 / 0.42 = 50,000 per QALY — exactly at the threshold; the choice of survival curve alone moves the result to the decision boundary.

• Re-run with 0% discounting: 24,000 / 0.60 = 40,000 per QALY — below the threshold; undiscounted future QALYs grow faster than future costs here, so this scenario is more favorable.

• Mark each row against the $50,000 line and report all four ICERs, flagging that the conclusion is robust to discounting but sensitive to the time horizon and the extrapolation family.

Result

Base-case ICER = 22,000 / 0.50 = 44,000 per QALY (cost-effective at $50,000). Scenarios: 10-year horizon 18,000 / 0.30 = 60,000 (flips the decision), Gompertz extrapolation 21,000 / 0.42 = 50,000 (exactly at the threshold), 0% discounting 24,000 / 0.60 = 40,000 (robust). The decision hinges on horizon and extrapolation — exactly the structural choices a PSA would not have tested.

Runnable example

import pandas as pd

WTP = 50_000.0

BASE = {"inc_cost": 22_000.0, "inc_qaly": 0.50}

# parameter -> (low_output, high_output) where each output is (inc_cost, inc_qaly)
# produced by re-running the model at the parameter's justified low/high value.
RANGES = {
    "drug_price":       ((18_000.0, 0.50), (28_000.0, 0.50)),
    "utility_gain":     ((22_000.0, 0.38), (22_000.0, 0.62)),
    "relapse_rr":       ((20_500.0, 0.44), (23_500.0, 0.56)),
}

# named structural scenarios -> model outputs from full re-runs
SCENARIOS = {
    "base_case_lifetime_3pct": (22_000.0, 0.50),
    "horizon_10_years":        (18_000.0, 0.30),
    "gompertz_extrapolation":  (21_000.0, 0.42),
    "discount_0pct":           (24_000.0, 0.60),
}

def icer(inc_cost: float, inc_qaly: float) -> float:
    return inc_cost / inc_qaly

def one_way_dsa() -> pd.DataFrame:
    base_icer = icer(**BASE)
    rows = []
    for param, (lo, hi) in RANGES.items():
        lo_icer, hi_icer = icer(*lo), icer(*hi)
        rows.append({
            "parameter": param,
            "icer_low": lo_icer, "icer_high": hi_icer,
            "swing": abs(hi_icer - lo_icer),            # tornado bar length
            "crosses_wtp": (min(lo_icer, hi_icer) <= WTP <= max(lo_icer, hi_icer)),
        })
    out = pd.DataFrame(rows).sort_values("swing", ascending=False)  # tornado order
    out.attrs["base_icer"] = base_icer
    return out

def scenario_table() -> pd.DataFrame:
    rows = [{"scenario": name, "inc_cost": c, "inc_qaly": q,
             "icer": icer(c, q), "cost_effective_at_wtp": icer(c, q) <= WTP}
            for name, (c, q) in SCENARIOS.items()]
    return pd.DataFrame(rows)

library(data.table)

WTP <- 50000

icer <- function(inc_cost, inc_qaly) inc_cost / inc_qaly

# parameter -> low/high model outputs (inc_cost, inc_qaly) from re-runs at justified bounds
ranges <- list(
  drug_price   = list(low = c(18000, 0.50), high = c(28000, 0.50)),
  utility_gain = list(low = c(22000, 0.38), high = c(22000, 0.62)),
  relapse_rr   = list(low = c(20500, 0.44), high = c(23500, 0.56))
)

scenarios <- list(
  base_case_lifetime_3pct = c(22000, 0.50),
  horizon_10_years        = c(18000, 0.30),
  gompertz_extrapolation  = c(21000, 0.42),
  discount_0pct           = c(24000, 0.60)
)

one_way_dsa <- function() {
  out <- rbindlist(lapply(names(ranges), function(p) {
    lo <- icer(ranges[[p]]$low[1],  ranges[[p]]$low[2])
    hi <- icer(ranges[[p]]$high[1], ranges[[p]]$high[2])
    data.table(parameter = p, icer_low = lo, icer_high = hi,
               swing = abs(hi - lo),
               crosses_wtp = min(lo, hi) <= WTP & WTP <= max(lo, hi))
  }))
  setorder(out, -swing)   # tornado order: widest swing on top
  out
}

scenario_table <- function() {
  rbindlist(lapply(names(scenarios), function(s) {
    ic <- icer(scenarios[[s]][1], scenarios[[s]][2])
    data.table(scenario = s,
               inc_cost = scenarios[[s]][1], inc_qaly = scenarios[[s]][2],
               icer = ic, cost_effective_at_wtp = ic <= WTP)
  }))
}