AHRQ EPC Methods Guide for Effectiveness and Comparative Effectiveness Reviews

What it is

The AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews (commonly the "AHRQ EPC Methods Guide" or "CER Methods Guide") is the master methodological reference maintained by the Agency for Healthcare Research and Quality (AHRQ) through its Effective Health Care (EHC) Program and the network of Evidence-based Practice Centers (EPCs). It is a living, chapter-based guide — launched in the AHRQ Series of Journal of Clinical Epidemiology papers (2010) and continuously updated as standalone EHC chapters — that prescribes how an EPC should run a comparative effectiveness review (CER): formulating the question with the PICOTS framework, searching and selecting evidence, assessing the risk of bias of individual studies, synthesizing results (qualitatively and via quantitative/meta-analytic methods), grading the strength of a body of evidence across the domains of study limitations, directness, consistency, precision, and reporting bias, and rating the applicability of the findings. It is a process-and-appraisal guide for evidence synthesis, not a primary-study reporting checklist and not a risk-of-bias instrument in itself.

When to use

Use the AHRQ EPC Methods Guide when you are producing or reviewing a systematic review / comparative effectiveness review of an intervention or exposure, especially one commissioned under or modeled on the EHC Program (EPC technical briefs, full CERs, USPSTF evidence reviews, CMS coverage evidence reviews). It is the governing standard when a review must (1) compare ≥2 interventions head-to-head, (2) integrate evidence of mixed design — RCTs plus observational/RWE studies drawn from claims, EHR, or registries — and (3) deliver a defensible strength-of-evidence grade for decision makers. Decision rules for which guide applies: use AHRQ EPC Methods Guide for the conduct and appraisal logic of an EHC-style CER; use PRISMA 2020 to report the systematic review and PRISMA-P to register its protocol; use ROBINS-I (which has superseded AHRQ's earlier RTI Item Bank approach) for per-study risk-of-bias of non-randomized studies cited within the CER; use a GRADE-based scheme or AHRQ's own strength-of-evidence grading for the certainty rating. For the primary observational studies you are appraising, the relevant reporting standards are STROBE / RECORD-PE / HARPER — not this guide.

What it requires

The guide enforces a structured chain of methodological decisions, each of which becomes the lens through which included RWE studies are appraised: - Question framing & scope — an explicit PICOTS statement (Population, Intervention, Comparator, Outcomes, Timing, Setting) and analytic framework linking interventions to intermediate and health outcomes. - Evidence search & selection — comprehensive, reproducible, dual-reviewer searching with pre-specified eligibility, including gray literature and regulatory/registry sources. - Risk-of-bias appraisal of included studies — design-appropriate assessment; for non-randomized/RWE evidence this means scrutinizing confounding control, exposure and outcome misclassification, selection/immortal-time bias, and time-zero alignment as reported by the primary study (the CER judges whether the study handled these, it does not itself analyze patient-level data). - Data-fitness and applicability — whether the underlying data source and population support the question, and how well the body of evidence applies to the target decision (applicability/PICOTS match). - Quantitative synthesis — when pooling is appropriate, pre-specified meta-analytic methods, heterogeneity assessment, and handling of sparse or observational data; otherwise a structured qualitative synthesis. - Strength-of-evidence grading — a transparent rating across study limitations, directness, consistency, precision, and reporting/publication bias, yielding High / Moderate / Low / Insufficient. - Sensitivity & bias analysis — assessment of how robust conclusions are to study-level bias, including reporting bias and, where included RWE warrants it, quantitative bias considerations.

When NOT to use — limitations and common misapplications

- It is not a primary-study reporting checklist. Authors of an individual claims/EHR cohort study should report with STROBE/RECORD-PE/HARPER and register a protocol; the AHRQ guide governs the review of such studies, not their conduct. Treating it as a single-study checklist is a category error a senior reviewer will flag immediately. - It is not, by itself, a risk-of-bias instrument or a quality score. It directs you to design-appropriate tools (now ROBINS-I for non-randomized studies); it does not produce a numeric quality score, and a high strength-of-evidence grade is a statement about the body of evidence, not a guarantee that any included observational study is causal. - It is not an HTA reference case or an economic-evaluation standard. For value/cost-effectiveness or payer decision frameworks, use NICE / CADTH / ICER reference cases and CHEERS for economic reporting — AHRQ EPC reviews inform but do not replace these. - Completing the steps does not make the underlying observational evidence causal. Synthesis discipline cannot repair confounding or selection bias baked into the primary RWE studies. - Checklist-as-theater — going through the chapters without the dual-review rigor, pre-specification, and honest strength-of-evidence grading they demand produces a CER in name only.

How it maps to this catalog

The AHRQ guide is a wrapper that, when a CER ingests real-world evidence, cross-walks directly onto this catalog's implementing concepts: - Question framing → picots-framework-rwe and comparative-effectiveness-research-cer-methods / cer-observational; the review object itself is a systematic-review (with meta-analysis-obs when pooling observational studies). - Data fitness-for-use → fit-for-purpose-data-assessment-rwe (does the source support the question?). - Appraising whether an included RWE study controlled confounding and aligned time zero → target-trial-emulation, active-comparator-new-user, high-dimensional-propensity-score-hdps-rwe, and time-zero-index-date-alignment-rwe. - Appraising estimand clarity and outcome/exposure misclassification in cited studies → estimands-ate-att-intercurrent-events-rwe, diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe, and claims-outcome-algorithm-ppv-sensitivity-rwe. - Appraising attrition/missing data in cited studies → attrition-and-loss-to-follow-up-rwe. - Strength-of-evidence robustness and sensitivity/quantitative bias analysis → quantitative-bias-analysis-toolkit-rwe and e-value-sensitivity-analysis.

Applied note (claims/EHR/registry RWE)

When an EHC-style CER pulls a claims- or EHR-based cohort study into its evidence base, the guide does not ask the reviewer to re-run the analysis — it asks the reviewer to judge, from what the primary study reports, whether time zero was aligned at initiation, whether an active comparator and new-user design controlled confounding by indication, whether the outcome phenotype was validated (PPV/sensitivity), and whether attrition was differential. Those judgments then feed the study-limitations domain of the strength-of-evidence grade. This is the practical seam between the AHRQ synthesis layer and the RWE-method concepts catalogued here.