ISPOR SUITABILITY Checklist
ISPOR good-practices checklist for assessing whether real-world data from electronic health records are trustworthy and fit-for-purpose to inform health technology assessment, structured around data delineation (characteristics, provenance, governance) and data fitness (reliability, relevance).
What it is
— The SUITABILITY checklist (Fleurence et al., Value in Health 2024) is a good-practices report of an ISPOR Task Force that gives HTA bodies, payers, and evidence generators a structured way to judge whether real-world data drawn from electronic health records (EHRs) are trustworthy enough, and relevant enough, to inform a health technology assessment. It is maintained by ISPOR — The Professional Society for Health Economics and Outcomes Research as part of its Good Practices series. The checklist is organized into two complementary arms. Data delineation establishes what the data are and whether they can be trusted, through three blocks: (1) data characteristics (structure, variables, coding systems, longitudinality, population captured); (2) data provenance (how the data were generated, transformed, curated, and versioned from the point of care to the analytic dataset); and (3) data governance (access, privacy, ethical and regulatory controls, and reproducibility). Data fitness-for- purpose then asks whether those data can actually answer the decision question, through (4) reliability (accuracy, completeness, plausibility, and consistency of the key variables) and (5) relevance (whether the population, exposures, outcomes, follow-up, and time window match the HTA question). SUITABILITY is a data-assessment instrument: it interrogates the substrate, not the analysis.
When to use
— Use SUITABILITY when an HTA submission, payer dossier, or value assessment rests on EHR-derived real-world evidence and a reviewer or generator must defend (or scrutinize) the data source before trusting any effect estimate. It is built for the HTA/payer decision context — NICE-style technology appraisals, CADTH/INESSS reviews, joint EU HTA clinical assessments, and the cost-effectiveness or comparative-effectiveness analyses that feed them — and for peer-reviewed reporting of EHR-based HTA evidence. Decision rule for which tool applies: reach for SUITABILITY specifically when the dossier's evidence comes from EHR data feeding an HTA decision. If the data are claims-only or registry-only, or the question is regulatory rather than HTA, a sibling instrument fits better — the ISPOR RWD questionnaire for general RWD source vetting, ENCePP / GVP Module VIII / ISPE- SCOPE for pharmacoepidemiologic and regulatory PASS work, and NICE's RWE framework as the HTA-body's own data-suitability expectations. SUITABILITY does not replace those; it is the EHR-for-HTA-specialized lens.
What it requires
— Substantively, SUITABILITY forces documentation of: the data model and variable provenance end-to-end (raw EHR → ETL/curation → analytic dataset, with versioning); governance and reproducibility of the data pipeline; data-fitness-for-use evidence — the reliability of the variables that carry the analysis (key exposures, outcomes, covariates) including completeness, plausibility, and internal consistency; and relevance of the captured population, time window, and endpoints to the HTA question. Framed for real-world data, that means it expects: explicit phenotype / algorithm definitions for diseases, exposures, and outcomes, with validation evidence (PPV, sensitivity) where the variable drives the result; clear handling of observable time and attrition / loss to follow-up in a visit-driven EHR; transparent treatment of missing data; and alignment of the data window with time-zero and the intended estimand. It does not itself prescribe the confounding- control or analytic strategy — but it asks whether the data can support the one chosen.
When NOT to use — limitations and common misapplications
— SUITABILITY is a fit-for-purpose data-assessment checklist, NOT a risk-of-bias instrument and NOT a study-quality score. Passing it tells you the EHR data are usable; it tells you nothing about whether the design is sound or the estimate causal — a SUITABILITY-clean dataset analyzed with a prevalent-user, immortal-time-ridden design is still biased. Concrete failure modes: (1) Wrong substrate — applying it to a claims-only or registry-only study where the ISPOR RWD questionnaire, ENCePP, or ISPE-SCOPE is the correct tool; SUITABILITY's provenance/curation logic is tuned to the messy, unstructured, visit-driven nature of EHRs. (2) Wrong decision context — treating it as a regulatory (FDA/EMA) submission standard; it is HTA/payer turf, and FDA's RWD/EHR-claims guidance or EMA/GVP govern the regulatory lane. (3) Checklist-as-theater — ticking boxes without producing the underlying validation/provenance evidence, which defeats the purpose. (4) Substituting it for design rigor — using a completed SUITABILITY assessment as if it certified the analysis; it must be paired with sound design (target-trial emulation, active-comparator new-user) and bias analysis. (5) Conflating reliability with relevance — accurate data for the wrong population or time window still fails fitness-for-purpose.
How it maps to this catalog
— SUITABILITY's two arms route directly to implementing concepts. The overarching fitness judgment is operationalized by fit-for-purpose-data-assessment-rwe. Data delineation → characteristics/provenance/governance: ohdsi-cdm (a common data model that makes characteristics and provenance auditable and reproducible) and continuous-enrollment-observable-time-rwe (defining observable person-time so "what the data capture" is explicit). Data fitness → reliability: variable accuracy is implemented by ehr-phenotyping-algorithms-rwe and diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe for cohort/condition definitions, claims-outcome-algorithm-ppv-sensitivity-rwe for the validation metrics (PPV, sensitivity) the checklist expects, attrition-and-loss-to-follow-up-rwe for completeness over follow-up, and missing-data-pattern-table-rwe for missingness transparency. Data fitness → relevance: matching data to the decision question is carried by estimands-ate-att-intercurrent-events-rwe (does the data support the target estimand and time-zero?) and, once relevance is established, the analytic engines that turn fit data into defensible estimates — target-trial-emulation, active-comparator-new-user, high-dimensional-propensity-score-hdps-rwe, and empirical-calibration-negative-controls-rwe for residual-confounding diagnostics. Use claims-analysis as the contrast: when the source is claims rather than EHR, that concept (with the ISPOR RWD questionnaire) is where the suitability assessment lives instead.
Applied note (EHR for HTA)
In a NICE-style appraisal built on a US/EU EHR network, SUITABILITY would have the submitter document the ETL from source EHR to analytic dataset with versions (provenance), show the governance and re-run path (governance), report PPV/sensitivity for the outcome phenotype against chart review (reliability), quantify loss to follow-up when patients leave the health system (reliability/relevance), and demonstrate that the captured population, line of therapy, and time horizon match the appraisal's decision problem (relevance) — before the committee weighs the comparative-effectiveness estimate at all.