ISPOR-ISPE Good Practices for Real-World Data Studies (Berger et al. 2017)

What it is

The Good Practices for Real-World Data Studies of Treatment and/or Comparative Effectiveness recommendations are the consensus output of the joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making (Berger et al., 2017), published simultaneously in Value in Health and Pharmacoepidemiology and Drug Safety. ISPOR (the Professional Society for Health Economics and Outcomes Research) and ISPE (the International Society for Pharmacoepidemiology) jointly maintain the document. It is a good-practice / design-and-conduct framework — not a numbered reporting checklist like STROBE/RECORD-PE and not a risk-of-bias instrument like ROBINS-I. Its central thesis is that real-world evidence can be decision-grade if and only if the study process is credible and reproducible, so the recommendations are organized around (1) a priori hypothesis testing versus exploratory analysis, (2) public study registration and pre-specified protocols, and (3) transparent, auditable implementation. The Task Force deliberately split its work: this good-practices statement covers what makes an RWD study trustworthy, while its companion, Wang, Schneeweiss, Berger et al. (2017) "Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0," provides the structured reporting template that operationalizes those principles for claims/EHR studies. Treat the two papers as one toolkit: the recommendations set the standard, the V1.0 template enforces it.

When to use

Apply this framework whenever a non-interventional study of treatment effects or comparative effectiveness built on routinely collected data (claims, EHR, registries, linked, or multi-database networks) is intended to inform a decision — regulatory (FDA/EMA), HTA/payer dossier, formulary/coverage, or a high-impact peer-reviewed manuscript. It is the right reference at the planning stage of any hypothesis-evaluating RWD study and again at the governance stage when you must demonstrate that the analysis was pre-specified rather than data-dredged. Decision rules for choosing THIS framework over siblings: use the ISPOR-ISPE good practices to set the overarching credibility expectations (registration, pre-specification, fitness-for-purpose, sensitivity analysis) for a hypothesis-evaluating treatment-effect study; switch to HARPER / START-RWE when you need a fill-in protocol/structured template for the design itself; use STROBE + RECORD-PE for the final manuscript's reporting items and flow diagram; use the ENCePP Guide/Checklist and EU PAS Register when the study is an EU PASS or otherwise falls under GVP Module VIII; and consult the FDA RWE Framework and FDA's RWD/EHR-claims guidance for US regulatory submissions. The ISPOR-ISPE document sits above the reporting checklists: it tells you how to make the study believable; the checklists tell you how to write it up.

What it requires

The recommendations enforce a specific set of credibility-bearing practices, and the V1.0 reporting template makes each one concrete for real-world data: - A priori declaration of intent. State whether the study is hypothesis-evaluating (confirmatory) or exploratory, and post the protocol and analysis plan to a public study register before looking at outcome data. Post-hoc, register-after-the-fact analyses are explicitly disfavored for decision-grade claims. - Design transparency. Fully specify the study design (cohort, case-control, self-controlled), eligibility, exposure and comparator definitions, time-zero/index-date alignment, follow-up, and the causal contrast/estimand — at a level of detail that allows independent re-execution. - Data fitness-for-use. Document the data source(s), why they are fit for the question (relevance and reliability), capture/lags, linkage, and provenance, before relying on them. - Operational definitions and phenotype/algorithm validation. Provide complete, versioned code lists and the validation evidence (PPV/sensitivity) for exposure, outcome, and covariate algorithms. - Confounding control and assumptions. Pre-specify the confounding-adjustment strategy (e.g., propensity or high-dimensional propensity methods, active-comparator new-user design) and test its assumptions (balance, positivity). - Attrition and missing data. Report the cohort-attrition cascade transparently and handle loss-to-follow-up and missingness explicitly. - Sensitivity and quantitative bias analysis. Pre-specify sensitivity analyses for key design choices and address residual/unmeasured confounding (e.g., negative controls, E-value). - Reproducibility. Tie every analytic decision to documented code and a protocol version so an independent team could reproduce the result — the explicit goal of the V1.0 template.

When NOT to use — limitations and common misapplications

- It is not a reporting checklist or a quality score. Do not treat the good-practices paper as a line-item box-ticking exercise the way you would STROBE or RECORD-PE; and the companion V1.0 template is a reproducibility/reporting structure, not a risk-of-bias instrument — it documents what was done, it does not grade internal validity. For risk-of-bias appraisal of a non-randomized study, use ROBINS-I (or ROBINS-E); for HTA suitability of an RWD source, use the ISPOR/ISPE suitability and RWD questionnaire tools. - Following the framework does not make a study causal. Registering a protocol and completing the V1.0 template documents transparency and reproducibility; it does not by itself remove confounding by indication, immortal-time bias, or selection bias. The design (active-comparator new-user, target-trial emulation) and the analysis carry the causal burden — the framework only makes those choices visible. - Checklist-as-theater. A registered protocol that is silently amended after seeing the data, or a completed template that points to unvalidated phenotypes, satisfies the letter and defeats the purpose. - Wrong tool for the design context. It does not cover RWE for medical devices (use FDA's device-specific RWE guidance), pragmatic/randomized real-world trials (use the FDA pragmatic-trials guidance, PRECIS-2, CONSORT-pragmatic), or EU-specific PASS procedural requirements (use ENCePP/EU PAS Register and GVP Module VIII). For purely descriptive RWE with no treatment-effect estimand, the treatment-effect/comparative-effectiveness emphasis here is heavier than needed.

How it maps to this catalog

Each requirement is implemented by a concrete concept in this repo: - A priori protocol/SAP and registration → `study-protocol-or-sap-elements`, `estimand-analysis-traceability-rwe`, `pass-imposed` / `pass-voluntary` (registration context). - Design transparency and the trial-like target → `target-trial-emulation`, `picots-framework-rwe`, `time-zero-index-date-alignment-rwe`. - Estimand and intercurrent events → `estimands-ate-att-intercurrent-events-rwe`. - Confounding control → `active-comparator-new-user`, `high-dimensional-propensity-score-hdps-rwe`, `propensity-score-methods-psm-iptw`, `dags-backdoor-criterion-drug-studies`. - Data fitness-for-use → `fit-for-purpose-data-assessment-rwe`, `claims-analysis`, `medicare-ffs-ma-commercial-claims-differences-rwe`, `continuous-enrollment-observable-time-rwe`. - Phenotype/algorithm validation → `diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe`, `claims-outcome-algorithm-ppv-sensitivity-rwe`, `algorithm-validation`. - Attrition and missing data → `attrition-and-loss-to-follow-up-rwe`, `database-feasibility-attrition-funnel-rwe`, `missing-data-pattern-table-rwe`. - Sensitivity / quantitative bias analysis → `e-value-sensitivity-analysis`, `quantitative-bias-analysis-toolkit-rwe`, `negative-control-outcomes-rwe`, `empirical-calibration-negative-controls-rwe`.

Applied note (claims/EHR/registry RWE)

For a comparative-effectiveness claims study, the framework converts to a concrete sequence: register a pre-specified protocol with the full estimand and analysis plan; document why the database is fit-for-purpose (`fit-for-purpose-data-assessment-rwe`, noting Medicare FFS vs MA capture gaps per `medicare-ffs-ma-commercial-claims-differences-rwe`); build an active-comparator new-user cohort with a justified washout and time-zero (`active-comparator-new-user`, `time-zero-index-date-alignment-rwe`); attach validated code lists and PPV evidence to every phenotype (`diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe`); control confounding with hdPS (`high-dimensional-propensity-score-hdps-rwe`); report the attrition funnel and covariate balance; and pre-specify sensitivity and negative-control analyses (`e-value-sensitivity-analysis`, `negative-control-outcomes-rwe`). The deliverable that satisfies the Task Force is a registered protocol plus a completed V1.0 reporting template that lets an independent team rerun the study from the documented code.