NICE Real-World Evidence Framework

What it is

The NICE Real-World Evidence (RWE) Framework is corporate guidance issued by the National Institute for Health and Care Excellence (reference ECD9, published 23 June 2022) that sets out how NICE expects real-world data and evidence to be planned, generated, assessed, and reported when it is used to inform NICE decisions — technology appraisals, highly specialised technologies, medical-technology and diagnostics guidance, and clinical guidelines. It is not a reporting checklist in the EQUATOR sense and not a single statistical method; it is an HTA-agency reference framework that pulls together three things: (1) a description of where RWE can add value in NICE decision making (characterising disease and treatment patterns, generating comparative-effect estimates where trials are absent or insufficient, validating surrogate-to-final-outcome links, and informing economic models); (2) a structured approach to assessing data suitability — the Data Suitability Assessment Tool (DataSAT) covering data provenance, governance, relevance, and quality (completeness, accuracy, validity, coverage); and (3) methodological expectations for the conduct of quantitative RWE studies of comparative effects, emphasising pre-specification, study registration, transparency, and explicit handling of confounding and bias. The framework is maintained by NICE (Science, Evidence and Analytics directorate) and is the UK HTA counterpart to the FDA RWE program guidances and the EMA/ENCePP and Heads of Medicines Agencies–EMA Big Data work; it is deliberately aligned in spirit with HARPER, STaRT-RWE, and the STROBE/RECORD-PE reporting tradition rather than replacing them.

When to use

Use the NICE RWE Framework whenever real-world data will feed a NICE (or, by extension, a UK/devolved-nation HTA) submission or appraisal — for example, a single-arm trial benchmarked against an external control drawn from registries or electronic health records; a comparative-effectiveness analysis substituting for a missing head-to-head trial; disease epidemiology, treatment pathways, or resource-use and cost inputs for the economic model; or surrogate-endpoint validation. Decision rules for when this framework applies rather than a sibling: choose the NICE framework when the decision context is HTA/reimbursement in the NICE remit and you need agency-specific expectations on data suitability and acceptability of comparative-effect estimates. Use the FDA RWE framework/guidances when the destination is a US regulatory (label/effectiveness) decision, and EMA/ENCePP guidance (ENCePP Methods/Checklist, GVP, registry-based-studies guideline) when the destination is an EU regulatory submission or an imposed PASS. These are complementary, not interchangeable: a single study can be designed to satisfy more than one, but the acceptance criteria and emphasis differ — NICE foregrounds transparency, data suitability, and fitness of the comparator/economic inputs for a UK decision problem (the PICO of the appraisal), whereas the regulatory frameworks foreground label-relevant effectiveness and pharmacovigilance.

What it requires

Framed for routinely collected data, the framework's substantive expectations cluster as: (1) Design transparency and pre-specification — a registered protocol and statistical analysis plan with the target estimand stated before analysis; NICE explicitly encourages study registration (e.g., ENCePP/EU PAS or equivalent) and reporting against an established guideline. (2) Data fitness-for-use — the DataSAT dimensions: provenance and governance (how data were collected and curated, legal basis), relevance (does the dataset's population, exposures, outcomes, and follow-up map to the decision PICO?), and quality (completeness, accuracy, internal/external validity, and coverage). (3) Phenotype and algorithm validation — operational definitions of exposures, outcomes, and covariates with validation evidence (e.g., PPV/sensitivity of code-based outcome and diagnosis algorithms). (4) Time-zero alignment — index-date definition that avoids immortal-time bias and aligns eligibility, treatment assignment, and the start of follow-up (the target-trial discipline). (5) Estimands and intercurrent events — explicit target population, treatment strategies, and handling of switching, discontinuation, and competing events. (6) Confounding control — appropriate methods (active-comparator new-user design, propensity-score and high-dimensional PS approaches, g-methods where time-varying confounding is present) with covariate-balance diagnostics. (7) Attrition and missing data — transparent attrition accounting and principled missing-data handling. (8) Sensitivity and quantitative bias analysis — negative controls, E-values, and probabilistic bias analysis to probe residual confounding, misclassification, and selection. The framework asks that each of these be justified for the specific decision problem, not merely performed.

When NOT to use — limitations and common misapplications

(1) It is not a reporting checklist and not a risk-of-bias score. Using the NICE framework where a reporting instrument is required (STROBE for observational studies, RECORD/RECORD-PE for routinely collected health data, CHEERS for economic evaluations) is a category error — the framework points to those tools; it does not substitute for them, and there is no "NICE framework score." (2) Completing DataSAT does not make a study causal or unbiased. A dataset can pass data-suitability review and still yield a confounded comparative estimate; suitability is necessary, not sufficient. (3) Framework-as-theater — appending a DataSAT table to a dossier without the design and analysis actually controlling confounding, or registering a protocol after analysis, defeats the purpose and is visible to NICE technical teams. (4) Wrong agency/context — do not present a NICE-framed dossier as if it discharges FDA or EMA expectations, or vice versa; the decision problem (UK PICO vs label population) differs. (5) Out of remit — it governs RWE used in NICE decisions; for pure regulatory effectiveness or pharmacovigilance, defer to FDA/EMA/ENCePP. (6) Not a license to skip the comparator question — using a non-comparable external control or a prevalent-user cohort because the data were "suitable" reintroduces the biases the methodological chapter warns against.

How it maps to this catalog

The framework's requirements are implemented by concrete concepts here: data fitness-for-use → fit-for-purpose-data-assessment-rwe (with payer/source nuance in medicare-ffs-ma-commercial-claims-differences-rwe and the claims-analysis base concept); design transparency and the decision PICO → picots-framework-rwe and study-protocol-or-sap-elements; the design backbone for comparative effects → target-trial-emulation and active-comparator-new-user; time-zero alignment → time-zero-index-date-alignment-rwe; phenotype/algorithm validation → diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe and claims-outcome-algorithm-ppv-sensitivity-rwe; estimands and intercurrent events → estimands-ate-att-intercurrent-events-rwe (with competing-risks-cause-specific-fine-gray-rwe for competing events); confounding control → high-dimensional-propensity-score-hdps-rwe, propensity-score-methods-psm-iptw, and marginal-structural-models-g-methods for time-varying confounding; attrition → attrition-and-loss-to-follow-up-rwe and database-feasibility-attrition-funnel-rwe; sensitivity and quantitative bias analysis → e-value-sensitivity-analysis, negative-control-outcomes-rwe, and quantitative-bias-analysis-toolkit-rwe. For the economic side that distinguishes NICE from regulators, the framework's model inputs map to survival-extrapolation-hta-rwe, cost-effectiveness, and healthcare-costs-pppm-pppy-pmpm. Applied note (claims/EHR/registry RWE): for a UK appraisal using an external control to benchmark a single-arm oncology trial, the practical NICE-aligned chain is — run DataSAT on the CPRD/registry source (relevance to the appraisal PICO, completeness of outcomes and mortality linkage), pre-register a target-trial-emulation protocol with an active-comparator-new-user contrast where feasible, fix time zero to avoid immortal time, balance with hdPS, account for attrition with a CONSORT-style funnel, and bound residual confounding with an E-value and negative controls before the estimate is allowed into the economic model.