GRADE

What it is

GRADE — Grading of Recommendations Assessment, Development and Evaluation is the dominant framework for two distinct tasks in evidence-based guideline development: (1) rating the certainty (formerly "quality") of a body of evidence for each outcome of interest, and (2) moving from that evidence to a graded strength of recommendation. It is maintained by the GRADE Working Group, an open international collaboration of guideline developers, methodologists, and clinicians, and is endorsed by Cochrane, the WHO, NICE, and dozens of professional societies and journals. GRADE rates certainty on four ordered levels — High, Moderate, Low, Very Low — reflecting confidence that the estimated effect is close to the true effect. Recommendations are rated as Strong or Conditional (weak). Critically, GRADE does not rate individual studies; it rates the aggregate evidence per outcome, consuming study-level risk-of-bias judgments (from ROBINS-I, RoB 2, etc.) as inputs. The framework's two reference artifacts are the Evidence Profile / Summary-of-Findings (SoF) table and, for recommendations, the Evidence-to-Decision (EtD) framework.

When to use

Use GRADE when you are synthesizing a body of evidence to inform a clinical practice guideline, a Cochrane or other systematic review, an HTA appraisal, or a payer/formulary recommendation — i.e., decision contexts where the unit of judgment is "how confident are we in the effect on this outcome, across all the evidence?" It applies to systematic reviews and meta-analyses (of RCTs, of observational studies, or mixed), network meta-analyses (via the NMA-specific GRADE guidance), and to the recommendations that guideline panels derive from them. Decision rule for which GRADE you apply: standard GRADE covers intervention effects on patient-important outcomes; questions of diagnostic test accuracy require the GRADE diagnostic extension, and prognosis questions require the prognosis adaptation — using core GRADE on these without the extension is a misapplication. GRADE is the right tool for HTA dossiers and journal-mandated certainty assessment; it is not a regulatory submission instrument (FDA and EMA do not adjudicate marketing decisions on GRADE), and it is not a reporting checklist (use PRISMA for the synthesis report, CHEERS for economic evaluations).

What it requires

GRADE enforces an explicit, domain-by-domain, auditable rating for each outcome: - Initial certainty by design. A body of RCT evidence starts at High; a body of observational (including real-world data) evidence starts at Low. - Five downgrading domains. Rate down for (1) risk of bias (limitations in the contributing studies — for RWE this is where confounding control, time-zero alignment, and exposure/outcome misclassification enter), (2) inconsistency (unexplained heterogeneity across studies), (3) indirectness (mismatch of population, intervention, comparator, or outcome — including surrogate vs patient-important endpoints — to the decision question), (4) imprecision (wide confidence intervals around the pooled estimate; optimal-information-size considerations), and (5) publication bias. - Three upgrading domains (observational evidence only). Rate up for a large magnitude of effect, a dose-response gradient, or plausible residual confounding that would bias toward the null (i.e., confounding that, if present, would shrink rather than inflate the observed effect). - Final certainty is one of High / Moderate / Low / Very Low, with each up- or down-grade documented and justified in the Evidence Profile / SoF table. - Strength of recommendation is determined separately through the EtD framework, weighing the balance of desirable and undesirable effects, the certainty of evidence, patients' values and preferences, resource use and cost-effectiveness, equity, acceptability, and feasibility. Strong vs Conditional reflects how confident the panel is that the desirable consequences outweigh the undesirable ones for most patients.

When NOT to use — limitations and common misapplications

- Rating a single study. GRADE rates a body of evidence per outcome. Applying the four certainty levels to one cohort study is a category error; for a single study's internal validity, use a risk-of-bias tool (ROBINS-I for non-randomized, RoB 2 for randomized). - GRADE is not a risk-of-bias instrument. Study-level bias is an input (the "risk of bias" downgrade domain), assessed with ROBINS-I/RoB 2; GRADE does not itself appraise individual studies. - Conflating certainty with strength of recommendation. They are separate axes and can legitimately diverge — a strong recommendation on low-certainty evidence is valid in defined "discordant" situations (e.g., life-saving intervention with little downside). Treating certainty as if it dictates recommendation strength misreads the framework. - Wrong question type without the extension. Diagnostic-accuracy and prognosis questions need the dedicated GRADE adaptations; core GRADE on these produces invalid ratings. - Checklist theater. Assigning "Low certainty — observational" by default, or ticking domains without explicit, transparent justification for each up/down move, defeats the purpose. The Evidence Profile must show the reasoning. - Treating GRADE as a reporting or RWD-design guideline. Phenotype validation, immortal-time avoidance, and estimand specification are study-conduct matters; they feed GRADE's risk-of-bias and indirectness judgments but are governed by STROBE/RECORD-PE, HARPER/STaRT-RWE, and the like — not by GRADE.

How it maps to this catalog

In an RWE evidence synthesis, the catalog's design and analysis concepts are not implemented by GRADE — they are the inputs that move GRADE's domains for an observational evidence body: - Risk-of-bias domain. A rigorous active-comparator, new-user design (active-comparator-new-user), target-trial emulation (target-trial-emulation), high-dimensional propensity scores (high-dimensional-propensity-score-hdps-rwe), and explicit immortal-time handling materially reduce the risk-of-bias concern — strong execution can justify not downgrading further, and in rare cases supports the "plausible-confounding-toward-the-null" upgrade. - Indirectness domain. Phenotype/algorithm validation (diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe) and fit-for-purpose data assessment determine whether the measured construct matches the PICO outcome; estimands and intercurrent-event handling (estimands-ate-att-intercurrent-events-rwe) determine whether the quantity estimated matches the decision question. Misalignment here is an indirectness downgrade. - Plausible-confounding upgrade / sensitivity. Quantitative bias analysis and the E-value speak directly to the "plausible residual confounding would bias toward the null" upgrade criterion and to how seriously to take the risk-of-bias downgrade. - Source-data context. Differences across claims, EHR, and registry data (e.g., claims-analysis, Medicare FFS vs MA coding intensity) affect outcome capture and transportability, again feeding the risk-of-bias and indirectness judgments.

Applied note (claims/EHR/registry RWE)

When a guideline panel grades an outcome supported only by claims- or EHR-based comparative studies, the body starts at Low. Document, per study, the confounding-control strategy and validation of the exposure/outcome phenotypes; decide the risk-of-bias downgrade on that basis (a well-executed ACNU target-trial emulation with validated phenotypes and negative-control diagnostics is a different animal from a prevalent-user, drug-vs-non-user claims analysis). Use a surrogate vs patient-important outcome check to set the indirectness rating, and bring an E-value or formal bias analysis to bear on whether residual confounding could plausibly explain — or, conversely, could only attenuate — the effect. Record every move in the Evidence Profile so the rating is reproducible and defensible to an HTA reviewer.