Scenario Analysis and Deterministic Sensitivity Analysis (DSA)

Scenario analysis and deterministic sensitivity analysis (DSA)

are the transparency workhorses of health-economic modeling. A one-way DSA changes a single input parameter from its base-case value to a justified low and high value while holding everything else fixed, records the resulting incremental cost-effectiveness ratio (ICER) or net monetary benefit (NMB), and repeats for each influential parameter; the results are ranked by output swing and displayed as a tornado diagram.

A multi-way DSA varies two or more parameters jointly (e.g., a two-way grid of drug price × treatment effect). A scenario analysis re-runs the entire model under a discrete alternative assumption set rather than a parameter range — a shorter or lifetime time horizon, 0%/3%/5% discount rates, an alternative survival-extrapolation family (Weibull vs Gompertz), a different perspective (payer vs societal), an alternative comparator, or a different source for a key input.

The two address different things: DSA probes parameter influence (which numeric inputs drive the result), while scenario analysis probes structural and methodological uncertainty (which modeling choices drive the result) — the layer that probabilistic sensitivity analysis (PSA) cannot reach, because PSA only propagates distributions within a fixed structure.

Core conceptual distinctions

(1) Influence vs decision uncertainty: DSA and scenarios answer "what matters and does the decision flip?"; PSA answers "given joint parameter uncertainty, how probable is it that the intervention is cost-effective?". They are complements mandated together by ISPOR-SMDM good practice, not substitutes.

(2) Parameter vs structural uncertainty: a parameter has a plausible numeric range (a utility, a cost, a relative risk) and belongs in DSA/PSA; a structural choice (horizon, extrapolation family, cycle length, competing-risk handling) has discrete alternatives and belongs in scenario analysis. Forcing structure into a parameter range (or ignoring it because the PSA looks tight) understates uncertainty.

(3) Justified ranges vs arbitrary ±20%: DSA ranges should come from confidence intervals, published extremes, or pre-specified plausibility arguments; a uniform ±20% applied to everything is a convention, not evidence, and the tornado it produces reflects the analyst's range choices as much as the data. (4) Threshold crossing: the decision-relevant output of any deterministic analysis is whether the ICER crosses the willingness-to-pay threshold (or NMB crosses zero) — a threshold analysis solves explicitly for the parameter value at which the decision flips.

Pros, cons, and trade-offs

(named against the alternatives).

• vs probabilistic sensitivity analysis (PSA): DSA/scenarios are transparent, cheap, and communicable — a reviewer can re-derive every number — and they expose structural uncertainty PSA ignores. But one-way DSA understates joint uncertainty (parameters move one at a time, correlations are ignored) and provides no probability statement. Run both: DSA/scenarios for influence and structure, PSA for decision uncertainty; HTA bodies (NICE, CADTH/CDA, ICER) expect all of them.
• vs tipping-point / threshold analysis: a threshold analysis is the inverted one-way DSA — instead of mapping a range to outputs, it solves for the input value where the decision changes. Prefer it when the question is "how wrong would this input have to be?"; prefer standard DSA when summarizing many parameters.
• One-way vs multi-way DSA: one-way is readable but ignores interactions; multi-way captures joint movement of two or three chosen parameters but becomes uninterpretable beyond that — past two or three dimensions the correct tool is PSA.
• Tornado diagram vs reporting all scenario rows: the tornado is an efficient influence ranking but hides asymmetry detail and invites range-gaming (wider stated range → longer bar → "more important"). A scenario table reporting incremental costs, incremental effects, and the resulting ICER per row is the auditable form; show both.

When NOT to use — and when it is actively misleading or dangerous

• As a substitute for PSA. A tornado diagram makes no probability statement; presenting deterministic ranges as if they characterized decision uncertainty ("the ICER ranged from X to Y, so we are confident...") is the classic misuse. Joint parameter uncertainty requires PSA.
• Cherry-picked scenarios. Running many scenarios and reporting only the favorable ones — or defining scenario assumptions after seeing results — converts an uncertainty analysis into advocacy. Pre-specify the scenario set (horizon, discounting, extrapolation, perspective) before running the model.
• Arbitrary or asymmetric-by-convenience ranges. ±20% on every parameter, or wide ranges only on parameters that favor the intervention, produce a tornado that misranks influence. Ranges need a stated source (CI, literature extremes, expert elicitation) reported next to the bar.
• One-way DSA on correlated parameters. Varying a survival parameter while holding its correlated counterpart fixed produces internally inconsistent model states (e.g., a shape parameter incompatible with the fixed scale); correlated blocks should move together (multi-way or PSA with correlation).
• Hiding the decision flip. Reporting only output ranges without flagging which scenarios cross the willingness-to-pay threshold buries the only result that matters to the decision maker; every scenario table should mark threshold crossings explicitly.

Data-source operational depth

Scenario analysis is where RWE inputs earn their keep — and where their fragility shows. In claims-informed models, standard scenarios include alternative cost sources (paid vs allowed amounts), alternative comorbidity/risk-adjustment specifications, and restricting to FFS person-time versus pooling Medicare Advantage encounters (whose paid amounts are unreliable).

In EHR/registry-informed models, scenarios swap the outcome algorithm (e.g., rwPFS definition variants) or the extrapolation family fitted to registry survival. In survey/utility inputs, scenarios swap the utility tariff or mapping algorithm.

Each scenario should change one named assumption, keep everything else at base case, and report the full incremental-cost / incremental-effect / ICER row so reviewers can trace exactly what moved.

Interpreting the output

The worked example shows a base-case ICER of $44,000/QALY (cost-effective at the $50,000 threshold), with the 10-year horizon scenario producing $60,000/QALY (which flips the decision) and 0% discounting producing $40,000/QALY (robust).

(1) Formal interpretation. A tornado diagram ranks each parameter by the width of the ICER swing it produces when moved from its lower to upper plausible bound, holding everything else at the base case. The widest bar identifies the single parameter whose uncertainty most drives the cost-effectiveness conclusion.

In this example, the time horizon — a structural assumption rather than a parameter — produces the largest swing and actually crosses the decision threshold: at 10 years the ICER rises to $60,000, which is above the $50,000 threshold.

Each scenario row should report the full set (incremental cost, incremental effect, ICER) so reviewers can verify the arithmetic and identify whether cost, effect, or both changed.

(2) Practical interpretation. When a scenario flips the cost-effectiveness verdict — as the 10-year horizon does here — it is a critical finding, not a minor sensitivity. The DSA has identified that the decision depends materially on whether the model follows patients for 5 versus 10 years, a structural choice that the PSA would not have tested (PSA samples from parameter distributions, not from alternative structural forms).

This finding should be prominently disclosed, and the rationale for the base-case horizon should be explicitly defended. Scenarios that keep the ICER well below the threshold across all tested assumptions (like 0% discounting here) provide evidence of robustness.

Decision diagram

flowchart TD
  M[Base-case model<br/>ICER / NMB] --> Q{Uncertainty type}
  Q -->|Numeric input with a range| D[One-way DSA<br/>justified low/high per parameter]
  Q -->|Discrete modeling choice| S[Scenario analysis<br/>horizon, discounting, extrapolation, perspective]
  Q -->|Joint parameter uncertainty| P[PSA<br/>separate concept]
  D --> T[Tornado diagram<br/>rank by output swing]
  T --> TP[Threshold analysis on top drivers<br/>input value where decision flips]
  S --> R[Scenario table<br/>inc cost, inc QALY, ICER per row]
  TP --> F{Any analysis crosses<br/>the WTP threshold?}
  R --> F
  F -->|no| Rob[Decision robust - report ranges]
  F -->|yes| Fra[Decision fragile - flag the assumption<br/>and the tipping value]

Worked example

Scenario

A cost-utility model finds a new treatment costs more but adds quality-adjusted life-years versus standard care, and the payer threshold is $50,000 per QALY. We compute the base-case ICER, then re-run the model under three pre-specified scenarios — a shorter 10-year time horizon, an alternative (Gompertz) survival extrapolation, and 0% discounting — and check which scenarios push the ICER across the threshold.

Dataset

Incremental cost and incremental QALYs from each model run, one row per pre-specified scenario.

Steps

Result

Base-case ICER = 22,000 / 0.50 = 44,000 per QALY (cost-effective at $50,000). Scenarios: 10-year horizon 18,000 / 0.30 = 60,000 (flips the decision), Gompertz extrapolation 21,000 / 0.42 = 50,000 (exactly at the threshold), 0% discounting 24,000 / 0.60 = 40,000 (robust). The decision hinges on horizon and extrapolation — exactly the structural choices a PSA would not have tested.

Trade-offs

Runnable example

import pandas as pd

WTP = 50_000.0

BASE = {"inc_cost": 22_000.0, "inc_qaly": 0.50}

# parameter -> (low_output, high_output) where each output is (inc_cost, inc_qaly)
# produced by re-running the model at the parameter's justified low/high value.
RANGES = {
    "drug_price":       ((18_000.0, 0.50), (28_000.0, 0.50)),
    "utility_gain":     ((22_000.0, 0.38), (22_000.0, 0.62)),
    "relapse_rr":       ((20_500.0, 0.44), (23_500.0, 0.56)),
}

# named structural scenarios -> model outputs from full re-runs
SCENARIOS = {
    "base_case_lifetime_3pct": (22_000.0, 0.50),
    "horizon_10_years":        (18_000.0, 0.30),
    "gompertz_extrapolation":  (21_000.0, 0.42),
    "discount_0pct":           (24_000.0, 0.60),
}

def icer(inc_cost: float, inc_qaly: float) -> float:
    return inc_cost / inc_qaly

def one_way_dsa() -> pd.DataFrame:
    base_icer = icer(**BASE)
    rows = []
    for param, (lo, hi) in RANGES.items():
        lo_icer, hi_icer = icer(*lo), icer(*hi)
        rows.append({
            "parameter": param,
            "icer_low": lo_icer, "icer_high": hi_icer,
            "swing": abs(hi_icer - lo_icer),            # tornado bar length
            "crosses_wtp": (min(lo_icer, hi_icer) <= WTP <= max(lo_icer, hi_icer)),
        })
    out = pd.DataFrame(rows).sort_values("swing", ascending=False)  # tornado order
    out.attrs["base_icer"] = base_icer
    return out

def scenario_table() -> pd.DataFrame:
    rows = [{"scenario": name, "inc_cost": c, "inc_qaly": q,
             "icer": icer(c, q), "cost_effective_at_wtp": icer(c, q) <= WTP}
            for name, (c, q) in SCENARIOS.items()]
    return pd.DataFrame(rows)

library(data.table)

WTP <- 50000

icer <- function(inc_cost, inc_qaly) inc_cost / inc_qaly

# parameter -> low/high model outputs (inc_cost, inc_qaly) from re-runs at justified bounds
ranges <- list(
  drug_price   = list(low = c(18000, 0.50), high = c(28000, 0.50)),
  utility_gain = list(low = c(22000, 0.38), high = c(22000, 0.62)),
  relapse_rr   = list(low = c(20500, 0.44), high = c(23500, 0.56))
)

scenarios <- list(
  base_case_lifetime_3pct = c(22000, 0.50),
  horizon_10_years        = c(18000, 0.30),
  gompertz_extrapolation  = c(21000, 0.42),
  discount_0pct           = c(24000, 0.60)
)

one_way_dsa <- function() {
  out <- rbindlist(lapply(names(ranges), function(p) {
    lo <- icer(ranges[[p]]$low[1],  ranges[[p]]$low[2])
    hi <- icer(ranges[[p]]$high[1], ranges[[p]]$high[2])
    data.table(parameter = p, icer_low = lo, icer_high = hi,
               swing = abs(hi - lo),
               crosses_wtp = min(lo, hi) <= WTP & WTP <= max(lo, hi))
  }))
  setorder(out, -swing)   # tornado order: widest swing on top
  out
}

scenario_table <- function() {
  rbindlist(lapply(names(scenarios), function(s) {
    ic <- icer(scenarios[[s]][1], scenarios[[s]][2])
    data.table(scenario = s,
               inc_cost = scenarios[[s]][1], inc_qaly = scenarios[[s]][2],
               icer = ic, cost_effective_at_wtp = ic <= WTP)
  }))
}

%let wtp = 50000;

/* ICER per DSA re-run, then swing per parameter (tornado bar length). */
data dsa;
  set work.dsa_runs;
  icer = inc_cost / inc_qaly;
run;

proc sql;
  create table tornado as
  select parameter,
         max(case when bound = 'low'  then icer end) as icer_low,
         max(case when bound = 'high' then icer end) as icer_high,
         abs(calculated icer_high - calculated icer_low) as swing,
         case when min(calculated icer_low, calculated icer_high) <= &wtp
               and &wtp <= max(calculated icer_low, calculated icer_high)
              then 1 else 0 end as crosses_wtp
  from dsa
  group by parameter
  order by swing desc;          /* tornado order: widest swing first */
quit;

/* Scenario table with explicit threshold flag. */
data scenario_table;
  set work.scenarios;
  icer = inc_cost / inc_qaly;
  cost_effective_at_wtp = (icer <= &wtp);
run;

proc print data=tornado;        run;
proc print data=scenario_table; run;

Citations

1. [1]Briggs AH. Handling uncertainty in cost-effectiveness models. PharmacoEconomics. 2000;17(5):479-500.
2. [2]Bilcke J, Beutels P, Brisson M, Jit M. Accounting for methodological, structural, and parameter uncertainty in decision-analytic models: a practical guide. Med Decis Making. 2011;31(4):675-692.
3. [3]Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making. 2012;32(5):722-732.

1. [4]Jain R, Grabner M, Onukwugha E. Sensitivity analysis in cost-effectiveness studies: from guidelines to practice. PharmacoEconomics. 2011;29(4):297-314.