Scoping Review

In plain language

A scoping review is a careful, by-the-rules survey of everything that has been published on a broad topic, built to answer "what's out there, and where are the holes?" rather than "does this drug work?" You register a search plan, run it across several databases, screen the hits down to the studies that fit, and then chart what each one did so you can draw a map of the field. The output is that map of who studied what, with which designs and data, and where nobody has looked yet, not a single combined number. The honest caveat: a thick, busy map tells you a lot of research exists, but it does not tell you the research is right.

A scoping review is a form of knowledge synthesis that addresses an exploratory question — "what is the extent, range, and nature of the evidence on X?" — rather than the focused, answerable effectiveness question of a systematic review. It follows the Arksey & O'Malley framework as refined by Levac and codified for reporting in PRISMA-ScR: (1) identify the research question; (2) identify relevant studies via a sensitive, pre-registered search; (3) select studies against explicit inclusion criteria, ideally with dual independent screening; (4) chart the data using a structured, piloted extraction form; (5) collate, summarize, and report results, typically as an evidence map; and (optional) (6) consult stakeholders. The deliverable is a map of the field — what has been studied, in which populations, with which designs and data sources, and where the white space is — not a pooled effect estimate.

Core conceptual distinction

The defining difference from a systematic review is the question type and the absence of synthesis-for-an-answer. A systematic review asks a narrow PICO question, mandates formal critical appraisal (e.g., RoB 2, ROBINS-I, Newcastle-Ottawa), and usually proceeds to a quantitative pooled estimate (meta-analysis) that drives a clinical or coverage decision. A scoping review deliberately keeps the question broad, does not require risk-of-bias appraisal of included studies, and does not pool effects — its output is descriptive (counts, evidence tables, bubble/heat maps). It is therefore a reconnaissance instrument: it scopes the size and shape of a body of evidence before anyone commits to the much heavier systematic review or HTA. It is not a "lighter systematic review" and must never be reported as if it answered an effectiveness question. Distinguish it also from a rapid review (a systematic review with deliberate methodological shortcuts under time pressure, which still targets an answerable question) and from a narrative/literature review (no protocol, no reproducible search).

Pros, cons, and trade-offs

(specific and comparative, naming the alternatives). - vs systematic review + meta-analysis: A scoping review can encompass heterogeneous designs, outcomes, and qualitative/grey literature that a meta-analysis cannot pool, and it surfaces the full landscape (e.g., every RWE design used to study a drug class). Cost: it yields no effect estimate and no graded certainty (no GRADE), so it cannot support a "drug A reduces event Y by Z%" claim or a reimbursement decision. Prefer a scoping review when the question is "what's out there and where are the gaps," and a systematic review when the question is "what is the effect." - vs narrative/expert literature review: A scoping review is protocol-driven, has a reproducible and reportable search, applies explicit and pre-specified inclusion criteria, and reports against PRISMA-ScR — so it is far less prone to selection and confirmation bias and is auditable. Cost: substantially more labor (often months, dual screening, charting form piloting). Prefer the scoping review whenever transparency and reproducibility matter (manuscript, regulatory dossier, grant background). - vs rapid review: A scoping review is broader in scope but typically does not appraise study quality; a rapid review keeps a focused question but cuts corners (single screener, restricted databases) to deliver fast. Prefer a scoping review when the goal is comprehensive mapping; prefer a rapid review when a decision-maker needs a defensible answer to a narrow question quickly. - Trade-off intrinsic to the design: by omitting risk-of-bias appraisal, a scoping review buys breadth and speed-per-study at the price of being silent on the credibility of what it maps — a strength when mapping, a danger when readers over-read the map as evidence of effect.

When to use

(clear decision rules). Choose a scoping review when: the question is broad or emerging and a precise PICO is not yet formulable; you need to identify the types of evidence and key concepts before designing a systematic review or registry study; you are mapping how a method or exposure has been operationalized across the literature (e.g., every washout/new-user variant used for an SGLT2-inhibitor safety question in claims); you must characterize a heterogeneous field that mixes RCTs, observational RWE, qualitative work, HTAs, and grey literature; or you are establishing the evidence-gap rationale for a grant, guideline, or HTA scoping document. Munn et al.'s decision logic: if the intended output is a yes/no answer about effectiveness/diagnostic accuracy, do a systematic review; if it is a map of the field, do a scoping review.

When NOT to use — and when it is actively misleading or dangerous

(clear decision rules). Do not use a scoping review when a decision hinges on the magnitude or certainty of an effect — payers, regulators, and guideline panels need appraisal and (usually) a pooled estimate, and a scoping review provides neither. It is actively misleading when its descriptive results are presented as evidence of effectiveness ("12 studies show benefit" conflates volume of literature with direction/certainty of effect — vote-counting is not synthesis). It is dangerous as the sole input to an HTA or label decision because it skips risk-of-bias appraisal: a field could be large yet uniformly biased (e.g., dozens of prevalent-user, immortal-time-afflicted claims studies), and the map would look reassuringly dense while every study is confounded. Do not use it when the question is already narrow and answerable — that is wasted breadth where a systematic review is the correct, more informative instrument. Finally, a scoping review that adds post hoc effect-direction conclusions has silently mutated into a low-quality systematic review without appraisal — the worst of both worlds.

Data-source operational depth

A scoping review's "data sources" are bibliographic, but in an RWE/HEOR context the included studies are themselves built on claims, EHR, registry, or linked data, and a credible review must chart and stratify by those substrates and their failure modes. - Bibliographic databases (the review's own data): A defensible search spans at least MEDLINE/PubMed, Embase, and one of the Cochrane/CINAHL/Web of Science set, plus grey literature (conference abstracts, HTA agency reports, ClinicalTrials.gov, regulator dossiers). Failure modes: relying on PubMed alone systematically misses Embase-indexed European pharmacoepidemiology; English-only restrictions drop relevant registry studies. Workaround: peer-review the search strategy (PRESS), report database + date of each search, and log every hit for the PRISMA-ScR flow. - Claims-based included studies: When charting a claims study, capture the database type because it drives interpretability. Medicare Advantage (MA)-only person-time lacks fee-for-service (FFS) claims, so a study built on MA enrollees may have incomplete encounter capture; chart whether the authors restricted to FFS/Parts A/B/D. Chart the exposure operationalization (NDC + `fill_date` + `days_supply`), washout length, and new-user vs prevalent-user status — the map is only useful if it exposes that, say, 18 of 22 included safety studies used prevalent-user cohorts (a systematic credibility flag the scoping map can legitimately surface without appraising each). - EHR-based included studies: Chart whether exposure was the order/administration vs a linked dispensing, and whether the study handled differential loss-to-follow-up when patients leave the system. Note structured-vs-NLP outcome ascertainment, which governs comparability across the map. - Registry / linked included studies: Chart adjudication of outcomes and disease-severity capture (registry strengths) and whether pharmacy exposure was linked from claims (a common registry weakness). For linked claims–EHR–vital-records studies, chart the linkage rate and any selection it induces. A frequent, chartable failure mode across elderly-cohort claims studies is differential competing risks by exposure (death competing with the event differs by arm) and immortal time in procedure studies (follow-up started before the index procedure); the scoping map should flag the prevalence of these design features even though it does not grade them.

Worked example (claims-style logic embedded in the inclusion criteria)

Question: map the landscape of US administrative-claims studies (2015–2025) comparing SGLT2 inhibitors with DPP-4 inhibitors for cardiovascular or renal outcomes, and characterize how each operationalized exposure and time zero. The claims-style rigor lives in the eligibility and charting rules, not in a statistical estimate: (1) Population/inclusion: peer-reviewed or HTA-report studies using ≥1 US claims database (e.g., Medicare FFS, MarketScan, Optum) with ≥1 cardiovascular or renal outcome. (2) Exposure-definition charting: for each study record `exposure_drug_class`, whether exposure was defined from pharmacy fills (NDC + `fill_date` + `days_supply`), the washout length (e.g., 365 days drug-free), and new-user vs prevalent-user status — the scoping map can then report that, e.g., only 14/31 studies used an active-comparator new-user design with a ≥365-day washout and continuous enrollment. (3) Data-source charting: record database type and, critically, whether the study excluded MA-only person-time (FFS claims completeness) and whether it required continuous medical + pharmacy enrollment across the washout — flagging the records where "no prior fill" could be unobserved missingness rather than a true washout. (4) Design-feature charting: record time-zero definition (first qualifying fill vs diagnosis date — the latter risks immortal time), competing-risk handling, and outcome validation. (5) Output: an evidence-map / bubble chart of study count by design × outcome, with the gap surfaced explicitly (e.g., "no linked claims–EHR study examined renal outcomes with a target-trial emulation in patients ≥75"). No effect is pooled; no study is risk-of-bias graded; the deliverable is the map and the gap — and an honest scoping review states plainly that the density of the map does not imply the SGLT2-vs-DPP4 effect is established.

Worked example

Scenario

You're mapping how US insurance-claims studies from 2015 to 2025 compared two diabetes drug classes (SGLT2 inhibitors vs. DPP-4 inhibitors) for heart and kidney outcomes. You are not trying to learn which drug is better; you want to see how many such studies exist and how each one defined who counts as 'treated.' The work is a funnel: you start with everything the databases return and narrow down, stage by stage, to the studies you actually include and chart. The table below is the running count an analyst tracks to fill in the PRISMA-ScR flow diagram.

Dataset

The PRISMA-ScR screening funnel: one row per stage, with the number of records moving through. These are the counts a reviewer logs as they go.

Steps

• Start at the top of the funnel: the databases (MEDLINE, Embase, Cochrane) plus grey literature return 1,240 records. The same paper often appears in more than one database, so 240 are duplicates; removing them leaves 1,000 unique records to actually look at.

• Title-and-abstract screening is the cheap first pass: two reviewers read just the titles and abstracts of all 1,000 records and drop anything clearly off-topic. Here 880 are excluded, so 1,000 - 880 = 120 records survive to the next stage.

• Full-text assessment is the careful second pass: you pull the full PDFs for those 120 and check each against the written inclusion rules (US claims data, a heart or kidney outcome, the right drug comparison). 89 fail on full read, so 120 - 89 = 31 studies are included.

• Now comes the part that makes this a scoping review and not a systematic review: instead of combining the 31 studies into one effect, you chart each one, recording how it defined exposure (a pharmacy fill vs. a diagnosis date), its washout length, and its study design. That charting is what lets you draw the map.

• The map reveals breadth and a gap. Of the 31 included studies, only 14 used the strongest design (an active-comparator new-user design with a year-long drug-free washout), and no study at all linked claims to EHR data for kidney outcomes in patients 75 and older, which is the white space a future study could fill.

• Read the map honestly: 31 studies exist and 14 are well-designed, but because you never appraised or pooled them, the density of the map says nothing about whether one drug class actually beats the other.

Result

The funnel is arithmetically consistent: 1,240 identified - 240 duplicates = 1,000 screened; 1,000 - 880 excluded at title/abstract = 120 assessed at full text; 120 - 89 excluded at full text = 31 studies included. The deliverable is the evidence map over those 31 studies (14/31 used the strongest design; one combination has zero studies) and the gap it exposes, not a pooled effect.

Runnable example

import pandas as pd
import numpy as np

def prisma_scr_flow(records: pd.DataFrame, screen: pd.DataFrame) -> dict:
    """PRISMA-ScR flow counts from retrieval through inclusion."""
    n_identified = len(records)
    n_unique = records.drop_duplicates(subset="doi").pipe(
        lambda d: len(d) + records["doi"].isna().sum())  # keep DOI-less rows; dedup only on DOI
    n_dup = n_identified - n_unique

    ta = screen[screen["stage"] == "title_abstract"]
    ft = screen[screen["stage"] == "full_text"]
    # A record is screened-in at a stage only if NO reviewer excluded it (conservative dual-screen rule).
    ta_in = ta.groupby("record_id")["decision"].apply(lambda d: (d == "include").all())
    ft_in = ft.groupby("record_id")["decision"].apply(lambda d: (d == "include").all())
    return {
        "identified": n_identified,
        "duplicates_removed": int(n_dup),
        "screened_title_abstract": int(ta["record_id"].nunique()),
        "excluded_title_abstract": int((~ta_in).sum()),
        "assessed_full_text": int(ft["record_id"].nunique()),
        "excluded_full_text": int((~ft_in).sum()),
        "included": int(ft_in.sum()),
    }

def screening_kappa(screen: pd.DataFrame, stage: str = "title_abstract") -> float:
    """Cohen's kappa for two reviewers' include/exclude decisions at a screening stage."""
    s = screen[screen["stage"] == stage]
    wide = (s.pivot_table(index="record_id", columns="reviewer", values="decision",
                          aggfunc="first").dropna())
    if wide.shape[1] != 2:
        raise ValueError("Cohen's kappa expects exactly two reviewers with overlapping decisions.")
    a, b = wide.iloc[:, 0], wide.iloc[:, 1]
    po = (a == b).mean()
    # Expected agreement under independence across the {include, exclude} categories.
    pe = sum((a == k).mean() * (b == k).mean() for k in pd.unique(pd.concat([a, b])))
    return float((po - pe) / (1 - pe)) if pe < 1 else 1.0

def evidence_map(chart: pd.DataFrame) -> pd.DataFrame:
    """Study count by design x outcome_domain -- the scoping deliverable (NOT a pooled effect)."""
    return (chart.pivot_table(index="design", columns="outcome_domain",
                              values="record_id", aggfunc="nunique", fill_value=0))

def credibility_flags(chart: pd.DataFrame) -> pd.Series:
    """Surface design-credibility prevalence the map should disclose (without grading any study)."""
    n = len(chart)
    return pd.Series({
        "n_included": n,
        "pct_prevalent_user": round(100 * (chart["design"] == "prevalent_user").mean(), 1),
        "pct_acnu_with_washout": round(100 * ((chart["design"] == "active_comparator_new_user")
                                              & (chart["washout_days"].fillna(0) >= 365)).mean(), 1),
        "pct_ma_only_excluded": round(100 * chart["ma_only_excluded"].mean(), 1),
        "pct_immortal_time_risk": round(100 * chart["time_zero"].isin(
            ["diagnosis_date", "procedure_date"]).mean(), 1),
        "pct_exposure_not_reported": round(100 * (chart["exposure_def"] == "not_reported").mean(), 1),
    })

library(data.table)

prisma_scr_flow <- function(records, screen) {
  setDT(records); setDT(screen)
  n_identified <- nrow(records)
  # Dedup only on DOI; keep DOI-less rows (a missing DOI is not a duplicate).
  n_unique <- uniqueN(records[!is.na(doi), doi]) + records[is.na(doi), .N]
  ta <- screen[stage == "title_abstract"]
  ft <- screen[stage == "full_text"]
  # A record is screened-in at a stage only if NO reviewer excluded it (conservative dual-screen rule).
  ta_in <- ta[, .(keep = all(decision == "include")), by = record_id]
  ft_in <- ft[, .(keep = all(decision == "include")), by = record_id]
  list(
    identified              = n_identified,
    duplicates_removed      = n_identified - n_unique,
    screened_title_abstract = uniqueN(ta$record_id),
    excluded_title_abstract = ta_in[keep == FALSE, .N],
    assessed_full_text      = uniqueN(ft$record_id),
    excluded_full_text      = ft_in[keep == FALSE, .N],
    included                = ft_in[keep == TRUE,  .N]
  )
}

screening_kappa <- function(screen, stage_name = "title_abstract") {
  setDT(screen)
  s <- screen[stage == stage_name]
  # One include/exclude per record per reviewer, then keep records both reviewers judged.
  w <- dcast(s, record_id ~ reviewer, value.var = "decision", fun.aggregate = function(x) x[1L])
  revs <- setdiff(names(w), "record_id")
  stopifnot(length(revs) == 2L)
  w <- w[complete.cases(w[, ..revs])]
  a <- w[[revs[1L]]]; b <- w[[revs[2L]]]
  po <- mean(a == b)
  cats <- union(unique(a), unique(b))
  pe <- sum(vapply(cats, function(k) mean(a == k) * mean(b == k), numeric(1)))
  if (pe >= 1) 1.0 else (po - pe) / (1 - pe)
}

evidence_map <- function(chart) {
  setDT(chart)
  # Study count by design x outcome_domain -- the scoping deliverable (NOT a pooled effect).
  dcast(unique(chart[, .(record_id, design, outcome_domain)]),
        design ~ outcome_domain, value.var = "record_id",
        fun.aggregate = function(x) uniqueN(x), fill = 0L)
}

credibility_flags <- function(chart) {
  setDT(chart); n <- nrow(chart)
  # Surface design-credibility prevalence the map should disclose -- without grading any single study.
  list(
    n_included                = n,
    pct_prevalent_user        = round(100 * mean(chart$design == "prevalent_user"), 1),
    pct_acnu_with_washout     = round(100 * mean(chart$design == "active_comparator_new_user" &
                                                 !is.na(chart$washout_days) & chart$washout_days >= 365), 1),
    pct_ma_only_excluded      = round(100 * mean(chart$ma_only_excluded), 1),
    pct_immortal_time_risk    = round(100 * mean(chart$time_zero %chin% c("diagnosis_date", "procedure_date")), 1),
    pct_exposure_not_reported = round(100 * mean(chart$exposure_def == "not_reported"), 1)
  )
}