Markov Transition Probabilities from Real-World Data

In plain language

A Markov model imagines a group of patients as a cohort moving between a small set of health states — such as stable disease, progressed disease, and death — one fixed time period at a time. A transition probability is the chance that a patient in one state ends up in a different (or the same) state at the end of that period, and these numbers come from watching real patients move through states in claims or electronic health record data. Every row of the resulting table of probabilities must add up to exactly 1, because each patient has to be somewhere at the end of each cycle. The hard part is that real-world data usually only captures a snapshot at each clinic visit, so estimating honest probabilities from irregular observations requires methods that account for what may have happened between visits.

A Markov (state-transition) cost-effectiveness model partitions a disease into a finite set of mutually exclusive, collectively exhaustive health states and moves a hypothetical cohort through them in fixed-length cycles. The numbers that drive everything the model produces — life-years, QALYs, and costs accrued in each state — are the transition probabilities: the per-cycle probability of moving from state i to state j. This concept is about estimating those probabilities from real-world longitudinal data (claims, EHR, registry, or linked sources), as opposed to lifting them from published trials or expert opinion. It is the data-engineering and statistical step that sits upstream of the decision model, not the economic evaluation itself.

Core conceptual distinction

Three things must be separated and pre-specified. (1) Probability vs rate. A transition rate is an instantaneous hazard (events per person-time, continuous); a transition probability is the chance of being in state j one cycle later (bounded 0–1, cycle-length-dependent). Cohort Markov models consume probabilities, but the honest way to get them from time-to-event data is to estimate rates and convert with the matrix exponential p = exp(Q·Δt), where Q is the transition-intensity matrix — not the naive 1 − exp(−rate·Δt), which is only correct for a single competing transition and silently mis-allocates probability when a state has more than one exit (Welton & Ades 2005). (2) Fully vs partially observed transitions. If you see every transition the moment it happens (e.g., a death index), counts give consistent estimates. If you observe state only at intermittent, irregular visits (the usual claims/EHR reality), the data are panel/interval-censored: an intervening state could have been missed, and count-ratio estimators are biased. Continuous-time multistate models (the `msm` framework, Jackson 2011) recover the intensity matrix from snapshots. (3) Time-homogeneity. The simplest model assumes a single Q for all cycles; real progression usually depends on time-in-state (semi-Markov / tunnel states) and calendar/age, so a homogeneous matrix can be badly wrong over a lifetime horizon. The estimand is therefore the cycle-specific transition-probability matrix for the target decision population, with uncertainty propagated (Dirichlet/multinomial draws for PSA per Briggs 2012), not a single point matrix.

Pros, cons, and trade-offs

- vs trial-derived or literature transition probabilities: RWD-derived transitions reflect the real treated population, settings, and adherence, and can be estimated for states and subgroups no trial reports. Cost: they inherit all of claims/EHR's measurement error (state misclassification, informative visit timing, left truncation), and naive count ratios from irregular data are biased. Prefer RWD when the decision population diverges from trial populations or when long-horizon/real-adherence transitions are needed — but validate against external benchmarks. - vs a partitioned-survival model (PSM): PSM reads area-under-the-curve directly from extrapolated PFS/OS curves and needs no transition structure, which makes it simple but structurally unconstrained — post-progression survival is implied, not modeled, and the three curves can imply impossible state occupancy. A Markov/multistate model enforces a coherent state structure and lets you model post-progression mortality explicitly. Prefer Markov when the disease has clinically meaningful intermediate states and back-transitions; prefer PSM for two-state oncology problems with mature survival curves where transition data are thin. - vs naive count-ratio (cohort-count) estimation: counting i→j moves and dividing by time-at-risk-in-i is trivial and transparent, but ignores interval censoring and competing risks, and cannot borrow strength across cells. A continuous-time multistate model (msm / Aalen-Johansen for the empirical matrix) is the methodologically correct upgrade at the cost of convergence fragility and stronger Markov assumptions. Prefer multistate estimation whenever states are observed only at irregular visits or death competes with progression.

When to use

Building a lifetime-horizon HTA cost-effectiveness model for a chronic, progressive disease with recognizable intermediate states (CKD stages, NYHA heart-failure classes, cancer remission/relapse, HIV CD4 strata) where you have longitudinal RWD spanning enough person-time to observe the transitions of interest, and where the decision population is not well represented by available trials. Also the natural tool when you need subgroup- or calendar-specific transition matrices, or when back-transitions (improvement, re-treatment) matter.

When NOT to use — and when it is actively misleading or dangerous

- Irregular observation treated as exact-time data. Plugging interval-censored claims snapshots into count-ratio formulas, or applying 1 − exp(−rate) per transition when a state has competing exits, produces matrices whose rows look fine but whose long-run state occupancy is wrong by the time it compounds over 40+ cycles. This is the most common and most dangerous error: it is invisible in one cycle and catastrophic in a lifetime model. - Disenrollment misread as a clinical transition. In claims, a patient who leaves the plan (especially to Medicare Advantage, where fee-for-service claims vanish) simply stops being observed. If censoring is not modeled, the cohort appears to "transition out," biasing every downstream probability. Never code end-of-data as a health-state move. - A two-state, no-back-transition problem. If the disease is really just alive→dead with one progression event and mature survival curves exist, a Markov model adds structure (and assumptions) you do not need — a partitioned-survival or simple survival extrapolation is more honest. - Sparse cells. When some i→j transitions have a handful of events, the matrix is unstable and PSA will swing wildly; Markov chaining propagates that instability. Collapse states or borrow strength (hierarchical/evidence synthesis, Welton & Ades 2005) rather than shipping a point matrix. - Strong time-in-state dependence forced into a homogeneous matrix. Diseases with accelerating hazards (e.g., late CKD) violate the memoryless assumption; a single homogeneous Q understates progression and overstates survival.

Data-source operational depth

- Claims (FFS vs MA): State must be derived per cycle from diagnosis/procedure/drug codes and any linked labs, because claims have no native "state" field. Pin a fixed cycle length, assign each person's state at each cycle boundary, and decide a missing-data rule (last-observation-carried-forward within a max gap vs treat as censored). Failure modes: Medicare Advantage enrollees lack FFS person-time, so apparent transitions are observation gaps — restrict to continuously FFS-enrolled spans (Parts A/B/D) and censor at MA switch. Differential disenrollment by health state (sicker patients change plans or die) makes "lost to follow-up" informative. Left truncation: prevalent patients enter mid-trajectory, so the apparent starting-state mix is not the inception mix. Death is poorly captured in claims alone — without a death index, the absorbing state leaks into "censored," which inflates survival; link to vital records or use a hospice/inpatient-discharge-disposition proxy and acknowledge its incompleteness. - EHR: States from labs/vitals/problem lists are richer (true eGFR for CKD staging, EF for HF) but visit-driven: a patient seen more often has more chances to be reclassified, so sicker patients accrue spurious transitions (ascertainment by visit frequency). External-care leakage means transitions happening outside the system are missed. Use the irregular visit times explicitly in a continuous-time model rather than forcing a fixed grid. - Registry: Disease-specific registries often record state directly and adjudicate it (cancer stage, dialysis start), which is ideal for the transition structure, but capture of competing mortality and of inter-state utilization cost may be weak — link to claims for costs and to a death index for the absorbing transition. - Linked claims–EHR–vital records: The ideal substrate (EHR-derived states + claims completeness + reliable death), but linkage selects the linkable subset (transportability) and introduces date discrepancies between lab, claim, and death dates that must be reconciled before assigning a state to a cycle boundary.

Worked claims example

Goal: a 3-month-cycle transition matrix for CKD progression (G3a → G3b → G4 → ESRD → Death, with possible back-transitions among the eGFR stages and Death/ESRD absorbing or near-absorbing) to feed a lifetime cost-utility model, using a linked claims+lab dataset. (1) Cohort/inception: adults with ≥2 eGFR values defining G3a/G3b and ≥365 days continuous FFS enrollment (Parts A/B) before the first qualifying eGFR (`index_date`), excluding Medicare Advantage person-time. (2) State assignment per cycle: lay a 91-day cycle grid from `index_date`; at each boundary assign the CKD stage from the most recent eGFR within the prior 180 days (carry forward); if no eGFR within the window and the person is still enrolled, treat the cycle as missing (interval-censored) rather than carrying an old value indefinitely. ESRD is set from the first dialysis/transplant procedure or revenue code (`dx`/`px` on the medical claim); Death from the linked vital-records `death_date`. (3) Censoring: stop person-time at the earliest of disenrollment, switch to MA, or end of data — these are censoring, never a transition to Death. (4) Estimation: because eGFR is observed at irregular lab dates, fit a continuous-time multistate (msm) model on the (person, eGFR-date, state) panel with Death as an exactly-timed absorbing state, recover the intensity matrix Q, and convert to a 91-day probability matrix via the matrix exponential; report the empirical Aalen-Johansen state-occupancy as a check. (5) Uncertainty: draw the transition matrix from the multivariate distribution of the fitted intensities (or Dirichlet draws of each row for a count-based sensitivity version) for probabilistic sensitivity analysis. (6) Diagnostics: compare modeled vs observed stage occupancy over follow-up, test time-homogeneity (split early vs late follow-up), and run a disenrollment-as-informative-censoring sensitivity analysis.

Interpreting the output

The worked example shows a 3-state Markov model (Stable, Progressed, Dead) with a 3-month cycle. After one cycle, a starting cohort of 1,000 in the Stable state redistributes to 585 Stable, 280 Progressed, and 135 Dead.

(1) Formal interpretation. Each row of the transition probability matrix must sum to 1.0, confirming that every patient in a state either stays or moves to exactly one other state — the matrix is stochastic. The result 585 + 280 + 135 = 1,000 confirms cohort conservation for this cycle. The 3-month cycle length means each probability represents the likelihood of a state change within a quarter; the same patient-level hazard translated to a 1-month cycle would yield different (generally smaller) per-cycle probabilities. A critical limitation is the Markov memorylessness assumption: the transition probability from Stable to Progressed is the same regardless of how many prior cycles the patient has already spent in Stable. When time in state is clinically meaningful (e.g., time since treatment start predicts progression risk), a tunnel-state extension, a multistate model with time-in-state covariates, or a DES approach is required.

(2) Practical interpretation. The 13.5% Dead probability from Stable over one 3-month cycle is the dominant driver of life-years in the model. Analysts should compare modeled state occupancy at each cycle against observed Kaplan-Meier curves or registry prevalence as a validation check — if the modeled progression curve diverges from the observed data by cycle 4, the transition probabilities need reestimation or the time-homogeneity assumption needs relaxing. Any PSA draws over the transition matrix must respect row-sum = 1 (use Dirichlet distributions for each row, not independent betas).

Worked example

Scenario

A health economist is building a simple cost-effectiveness model for a new cancer drug. She defines three health states: Stable (disease has not progressed), Progressed (disease has worsened), and Dead. Using one year of claims data she estimates the per-cycle (3-month) probabilities of moving between these states. She starts with a cohort of 1,000 patients and wants to see how they redistribute after one cycle.

Dataset

Transition probability matrix estimated from claims data. Each row is the origin state; each column is the destination state after one 3-month cycle. Every row sums to 1.0.

Steps

• Start with 1,000 patients split across states: 700 Stable, 250 Progressed, 50 Dead.

• Multiply each starting group by its row of transition probabilities to find contributions to each destination state.

• Stable patients contribute: 700 x 0.80 = 560 remain Stable; 700 x 0.15 = 105 move to Progressed; 700 x 0.05 = 35 move to Dead.

• Progressed patients contribute: 250 x 0.10 = 25 return to Stable; 250 x 0.70 = 175 remain Progressed; 250 x 0.20 = 50 move to Dead.

• Dead patients contribute: 50 x 1.00 = 50 stay Dead (absorbing state, no exits).

• Sum the contributions column by column to get the end-of-cycle cohort: Stable = 560 + 25 + 0 = 585; Progressed = 105 + 175 + 0 = 280; Dead = 35 + 50 + 50 = 135.

Result

After one 3-month cycle the cohort of 1,000 redistributes to 585 Stable, 280 Progressed, and 135 Dead. Total = 585 + 280 + 135 = 1,000, confirming the cohort is conserved. The model repeats this multiplication every cycle, chaining the matrix forward over the full time horizon to accumulate life-years and costs in each state.

Runnable example

import pandas as pd
import numpy as np

def empirical_transition_matrix(states: pd.DataFrame) -> pd.DataFrame:
    s = states.sort_values(["person_id", "cycle"]).copy()

    # Pair each observed cycle with the SAME person's next observed cycle.
    s["next_cycle"] = s.groupby("person_id")["cycle"].shift(-1)
    s["next_state"] = s.groupby("person_id")["state"].shift(-1)

    # Keep only adjacent cycles (no gap) so an unobserved intervening state
    # is not silently collapsed into a single transition.
    moves = s[s["next_cycle"] == s["cycle"] + 1]

    order = sorted(set(s["state"]))
    counts = (moves.groupby(["state", "next_state"])
                   .size()
                   .unstack(fill_value=0)
                   .reindex(index=order, columns=order, fill_value=0))

    row_tot = counts.sum(axis=1).replace(0, np.nan)
    P = counts.div(row_tot, axis=0)

    # Absorbing/unseen-exit rows: keep the cohort in-state (self-loop = 1).
    P = P.fillna(0.0)
    empty_rows = P.sum(axis=1) == 0
    for st in P.index[empty_rows]:
        P.loc[st, st] = 1.0
    return P  # rows sum to 1.0; pass to PSA via row-wise Dirichlet draws of `counts`

library(msm)
library(expm)

CYCLE_DAYS <- 91  # 3-month model cycle

# qmat: allowed transitions get a small positive initial intensity; 0 = structurally impossible.
# Example 5-state CKD: 1=G3a 2=G3b 3=G4 4=ESRD 5=Death (5 absorbing).
qmat <- rbind(
  c(0, 0.1, 0,   0,   0.02),
  c(0.05, 0, 0.1, 0,   0.03),
  c(0, 0.05, 0,  0.1,  0.05),
  c(0, 0,   0,   0,    0.10),
  c(0, 0,   0,   0,    0))

fit <- msm(state ~ obs_time, subject = person_id, data = panel,
           qmatrix = qmat, deathexact = 5, gen.inits = TRUE)

# Recover the estimated intensity matrix Q and convert to a CYCLE_DAYS probability matrix.
Q <- qmatrix.msm(fit, ci = "none")
P_cycle <- expm(Q * CYCLE_DAYS)          # rows sum to 1; this is the Markov-model input
# pmatrix.msm(fit, t = CYCLE_DAYS) gives the same matrix directly, with CIs for PSA.