Scoping Review

A scoping review is a form of knowledge synthesis that addresses an exploratory question — "what is the extent, range, and nature of the evidence on X?" — rather than the focused, answerable effectiveness question of a systematic review. It follows the Arksey & O'Malley framework as refined by Levac and codified for reporting in PRISMA-ScR:

1. identify the research question;
2. identify relevant studies via a sensitive, pre-registered search;
3. select studies against explicit inclusion criteria, ideally with dual independent screening;
4. chart the data using a structured, piloted extraction form;
5. collate, summarize, and report results, typically as an evidence map; and (optional)
6. consult stakeholders.

The deliverable is a map of the field — what has been studied, in which populations, with which designs and data sources, and where the white space is — not a pooled effect estimate.

Core conceptual distinction

The defining difference from a systematic review is the question type and the absence of synthesis-for-an-answer. A systematic review asks a narrow PICO question, mandates formal critical appraisal (e.g., RoB 2, ROBINS-I, Newcastle-Ottawa), and usually proceeds to a quantitative pooled estimate (meta-analysis) that drives a clinical or coverage decision.

A scoping review deliberately keeps the question broad, does not require risk-of-bias appraisal of included studies, and does not pool effects — its output is descriptive (counts, evidence tables, bubble/heat maps). It is therefore a reconnaissance instrument: it scopes the size and shape of a body of evidence before anyone commits to the much heavier systematic review or HTA.

It is not a "lighter systematic review" and must never be reported as if it answered an effectiveness question. Distinguish it also from a rapid review (a systematic review with deliberate methodological shortcuts under time pressure, which still targets an answerable question) and from a narrative/literature review (no protocol, no reproducible search).

Pros, cons, and trade-offs

(specific and comparative, naming the alternatives).

• vs systematic review + meta-analysis: A scoping review can encompass heterogeneous designs, outcomes, and qualitative/grey literature that a meta-analysis cannot pool, and it surfaces the full landscape (e.g., every RWE design used to study a drug class). Cost: it yields no effect estimate and no graded certainty (no GRADE), so it cannot support a "drug A reduces event Y by Z%" claim or a reimbursement decision. Prefer a scoping review when the question is "what's out there and where are the gaps," and a systematic review when the question is "what is the effect."
• vs narrative/expert literature review: A scoping review is protocol-driven, has a reproducible and reportable search, applies explicit and pre-specified inclusion criteria, and reports against PRISMA-ScR — so it is far less prone to selection and confirmation bias and is auditable. Cost: substantially more labor (often months, dual screening, charting form piloting). Prefer the scoping review whenever transparency and reproducibility matter (manuscript, regulatory dossier, grant background).
• vs rapid review: A scoping review is broader in scope but typically does not appraise study quality; a rapid review keeps a focused question but cuts corners (single screener, restricted databases) to deliver fast. Prefer a scoping review when the goal is comprehensive mapping; prefer a rapid review when a decision-maker needs a defensible answer to a narrow question quickly.
• Trade-off intrinsic to the design: by omitting risk-of-bias appraisal, a scoping review buys breadth and speed-per-study at the price of being silent on the credibility of what it maps — a strength when mapping, a danger when readers over-read the map as evidence of effect.

When NOT to use — and when it is actively misleading or dangerous

(clear decision rules). Do not use a scoping review when a decision hinges on the magnitude or certainty of an effect — payers, regulators, and guideline panels need appraisal and (usually) a pooled estimate, and a scoping review provides neither.

It is actively misleading when its descriptive results are presented as evidence of effectiveness ("12 studies show benefit" conflates volume of literature with direction/certainty of effect — vote-counting is not synthesis). It is dangerous as the sole input to an HTA or label decision because it skips risk-of-bias appraisal: a field could be large yet uniformly biased (e.g., dozens of prevalent-user, immortal-time-afflicted claims studies), and the map would look reassuringly dense while every study is confounded.

Do not use it when the question is already narrow and answerable — that is wasted breadth where a systematic review is the correct, more informative instrument. Finally, a scoping review that adds post hoc effect-direction conclusions has silently mutated into a low-quality systematic review without appraisal — the worst of both worlds.

Data-source operational depth

A scoping review's "data sources" are bibliographic, but in an RWE/HEOR context the included studies are themselves built on claims, EHR, registry, or linked data, and a credible review must chart and stratify by those substrates and their failure modes.

• Bibliographic databases (the review's own data): A defensible search spans at least MEDLINE/PubMed, Embase, and one of the Cochrane/CINAHL/Web of Science set, plus grey literature (conference abstracts, HTA agency reports, ClinicalTrials.gov, regulator dossiers). Failure modes: relying on PubMed alone systematically misses Embase-indexed European pharmacoepidemiology; English-only restrictions drop relevant registry studies. Workaround: peer-review the search strategy (PRESS), report database + date of each search, and log every hit for the PRISMA-ScR flow.
• Claims-based included studies: When charting a claims study, capture the database type because it drives interpretability. Medicare Advantage (MA)-only person-time lacks fee-for-service (FFS) claims, so a study built on MA enrollees may have incomplete encounter capture; chart whether the authors restricted to FFS/Parts A/B/D. Chart the exposure operationalization (NDC + `fill_date` + `days_supply`), washout length, and new-user vs prevalent-user status — the map is only useful if it exposes that, say, 18 of 22 included safety studies used prevalent-user cohorts (a systematic credibility flag the scoping map can legitimately surface without appraising each).
• EHR-based included studies: Chart whether exposure was the order/administration vs a linked dispensing, and whether the study handled differential loss-to-follow-up when patients leave the system. Note structured-vs-NLP outcome ascertainment, which governs comparability across the map.
• Registry / linked included studies: Chart adjudication of outcomes and disease-severity capture (registry strengths) and whether pharmacy exposure was linked from claims (a common registry weakness). For linked claims–EHR–vital-records studies, chart the linkage rate and any selection it induces. A frequent, chartable failure mode across elderly-cohort claims studies is differential competing risks by exposure (death competing with the event differs by arm) and immortal time in procedure studies (follow-up started before the index procedure); the scoping map should flag the prevalence of these design features even though it does not grade them.

Worked example (claims-style logic embedded in the inclusion criteria)

Question: map the landscape of US administrative-claims studies (2015–2025) comparing SGLT2 inhibitors with DPP-4 inhibitors for cardiovascular or renal outcomes, and characterize how each operationalized exposure and time zero. The claims-style rigor lives in the eligibility and charting rules, not in a statistical estimate:

(1) Population/inclusion: peer-reviewed or HTA-report studies using ≥1 US claims database (e.g., Medicare FFS, MarketScan, Optum) with ≥1 cardiovascular or renal outcome. (2) Exposure-definition charting: for each study record `exposure_drug_class`, whether exposure was defined from pharmacy fills (NDC + `fill_date` + `days_supply`), the washout length (e.g., 365 days drug-free), and new-user vs prevalent-user status — the scoping map can then report that, e.g., only 14/31 studies used an active-comparator new-user design with a ≥365-day washout and continuous enrollment.

(3) Data-source charting: record database type and, critically, whether the study excluded MA-only person-time (FFS claims completeness) and whether it required continuous medical + pharmacy enrollment across the washout — flagging the records where "no prior fill" could be unobserved missingness rather than a true washout.

(4) Design-feature charting: record time-zero definition (first qualifying fill vs diagnosis date — the latter risks immortal time), competing-risk handling, and outcome validation. (5) Output: an evidence-map / bubble chart of study count by design × outcome, with the gap surfaced explicitly (e.g., "no linked claims–EHR study examined renal outcomes with a target-trial emulation in patients ≥75").

No effect is pooled; no study is risk-of-bias graded; the deliverable is the map and the gap — and an honest scoping review states plainly that the density of the map does not imply the SGLT2-vs-DPP4 effect is established.

Decision diagram

flowchart TD
  ID[Identification: records from MEDLINE + Embase + Cochrane + grey literature<br/>n = identified] --> DEDUP[Remove duplicates<br/>n = duplicates removed]
  DEDUP --> TA[Title/abstract screening<br/>dual independent reviewers]
  TA -->|excluded| TAX[Excluded at title/abstract<br/>n with reasons]
  TA --> FT[Full-text assessment for eligibility<br/>against pre-specified criteria]
  FT -->|excluded| FTX[Excluded at full text<br/>n with reasons logged]
  FT --> INC[Included studies<br/>charted with piloted form]
  INC --> MAP[Collate + summarize -> evidence map<br/>counts by population x design x outcome; gaps]

flowchart TD
  Q{What does the decision-maker need?} -->|A map of what evidence exists and where the gaps are| SCOPE[Scoping review<br/>broad question, no risk-of-bias appraisal, no pooled effect]
  Q -->|A yes/no answer on effectiveness, safety magnitude, or accuracy| NARROW{Is a precise PICO formulable now?}
  NARROW -->|Yes, and time is adequate| SR[Systematic review<br/>formal appraisal + usually meta-analysis -> GRADE]
  NARROW -->|Yes, but a fast answer is needed| RR[Rapid review<br/>focused question, deliberate methodological shortcuts]
  NARROW -->|No, the field is too broad or emerging| SCOPE
  SCOPE --> GAP[Output: evidence map + research gaps -> informs a future systematic review or RWE study]
  SR --> DEC[Output: graded effect estimate -> guideline / HTA / label decision]

Worked example

Scenario

You're mapping how US insurance-claims studies from 2015 to 2025 compared two diabetes drug classes (SGLT2 inhibitors vs. DPP-4 inhibitors) for heart and kidney outcomes. You are not trying to learn which drug is better; you want to see how many such studies exist and how each one defined who counts as 'treated.' The work is a funnel: you start with everything the databases return and narrow down, stage by stage, to the studies you actually include and chart. The table below is the running count an analyst tracks to fill in the PRISMA-ScR flow diagram.

Dataset

The PRISMA-ScR screening funnel: one row per stage, with the number of records moving through. These are the counts a reviewer logs as they go.

Steps

Result

The funnel is arithmetically consistent: 1,240 identified - 240 duplicates = 1,000 screened; 1,000 - 880 excluded at title/abstract = 120 assessed at full text; 120 - 89 excluded at full text = 31 studies included. The deliverable is the evidence map over those 31 studies (14/31 used the strongest design; one combination has zero studies) and the gap it exposes, not a pooled effect.

Trade-offs

Runnable example

import pandas as pd
import numpy as np

def prisma_scr_flow(records: pd.DataFrame, screen: pd.DataFrame) -> dict:
    """PRISMA-ScR flow counts from retrieval through inclusion."""
    n_identified = len(records)
    n_unique = records.drop_duplicates(subset="doi").pipe(
        lambda d: len(d) + records["doi"].isna().sum())  # keep DOI-less rows; dedup only on DOI
    n_dup = n_identified - n_unique

    ta = screen[screen["stage"] == "title_abstract"]
    ft = screen[screen["stage"] == "full_text"]
    # A record is screened-in at a stage only if NO reviewer excluded it (conservative dual-screen rule).
    ta_in = ta.groupby("record_id")["decision"].apply(lambda d: (d == "include").all())
    ft_in = ft.groupby("record_id")["decision"].apply(lambda d: (d == "include").all())
    return {
        "identified": n_identified,
        "duplicates_removed": int(n_dup),
        "screened_title_abstract": int(ta["record_id"].nunique()),
        "excluded_title_abstract": int((~ta_in).sum()),
        "assessed_full_text": int(ft["record_id"].nunique()),
        "excluded_full_text": int((~ft_in).sum()),
        "included": int(ft_in.sum()),
    }

def screening_kappa(screen: pd.DataFrame, stage: str = "title_abstract") -> float:
    """Cohen's kappa for two reviewers' include/exclude decisions at a screening stage."""
    s = screen[screen["stage"] == stage]
    wide = (s.pivot_table(index="record_id", columns="reviewer", values="decision",
                          aggfunc="first").dropna())
    if wide.shape[1] != 2:
        raise ValueError("Cohen's kappa expects exactly two reviewers with overlapping decisions.")
    a, b = wide.iloc[:, 0], wide.iloc[:, 1]
    po = (a == b).mean()
    # Expected agreement under independence across the {include, exclude} categories.
    pe = sum((a == k).mean() * (b == k).mean() for k in pd.unique(pd.concat([a, b])))
    return float((po - pe) / (1 - pe)) if pe < 1 else 1.0

def evidence_map(chart: pd.DataFrame) -> pd.DataFrame:
    """Study count by design x outcome_domain -- the scoping deliverable (NOT a pooled effect)."""
    return (chart.pivot_table(index="design", columns="outcome_domain",
                              values="record_id", aggfunc="nunique", fill_value=0))

def credibility_flags(chart: pd.DataFrame) -> pd.Series:
    """Surface design-credibility prevalence the map should disclose (without grading any study)."""
    n = len(chart)
    return pd.Series({
        "n_included": n,
        "pct_prevalent_user": round(100 * (chart["design"] == "prevalent_user").mean(), 1),
        "pct_acnu_with_washout": round(100 * ((chart["design"] == "active_comparator_new_user")
                                              & (chart["washout_days"].fillna(0) >= 365)).mean(), 1),
        "pct_ma_only_excluded": round(100 * chart["ma_only_excluded"].mean(), 1),
        "pct_immortal_time_risk": round(100 * chart["time_zero"].isin(
            ["diagnosis_date", "procedure_date"]).mean(), 1),
        "pct_exposure_not_reported": round(100 * (chart["exposure_def"] == "not_reported").mean(), 1),
    })

library(data.table)

prisma_scr_flow <- function(records, screen) {
  setDT(records); setDT(screen)
  n_identified <- nrow(records)
  # Dedup only on DOI; keep DOI-less rows (a missing DOI is not a duplicate).
  n_unique <- uniqueN(records[!is.na(doi), doi]) + records[is.na(doi), .N]
  ta <- screen[stage == "title_abstract"]
  ft <- screen[stage == "full_text"]
  # A record is screened-in at a stage only if NO reviewer excluded it (conservative dual-screen rule).
  ta_in <- ta[, .(keep = all(decision == "include")), by = record_id]
  ft_in <- ft[, .(keep = all(decision == "include")), by = record_id]
  list(
    identified              = n_identified,
    duplicates_removed      = n_identified - n_unique,
    screened_title_abstract = uniqueN(ta$record_id),
    excluded_title_abstract = ta_in[keep == FALSE, .N],
    assessed_full_text      = uniqueN(ft$record_id),
    excluded_full_text      = ft_in[keep == FALSE, .N],
    included                = ft_in[keep == TRUE,  .N]
  )
}

screening_kappa <- function(screen, stage_name = "title_abstract") {
  setDT(screen)
  s <- screen[stage == stage_name]
  # One include/exclude per record per reviewer, then keep records both reviewers judged.
  w <- dcast(s, record_id ~ reviewer, value.var = "decision", fun.aggregate = function(x) x[1L])
  revs <- setdiff(names(w), "record_id")
  stopifnot(length(revs) == 2L)
  w <- w[complete.cases(w[, ..revs])]
  a <- w[[revs[1L]]]; b <- w[[revs[2L]]]
  po <- mean(a == b)
  cats <- union(unique(a), unique(b))
  pe <- sum(vapply(cats, function(k) mean(a == k) * mean(b == k), numeric(1)))
  if (pe >= 1) 1.0 else (po - pe) / (1 - pe)
}

evidence_map <- function(chart) {
  setDT(chart)
  # Study count by design x outcome_domain -- the scoping deliverable (NOT a pooled effect).
  dcast(unique(chart[, .(record_id, design, outcome_domain)]),
        design ~ outcome_domain, value.var = "record_id",
        fun.aggregate = function(x) uniqueN(x), fill = 0L)
}

credibility_flags <- function(chart) {
  setDT(chart); n <- nrow(chart)
  # Surface design-credibility prevalence the map should disclose -- without grading any single study.
  list(
    n_included                = n,
    pct_prevalent_user        = round(100 * mean(chart$design == "prevalent_user"), 1),
    pct_acnu_with_washout     = round(100 * mean(chart$design == "active_comparator_new_user" &
                                                 !is.na(chart$washout_days) & chart$washout_days >= 365), 1),
    pct_ma_only_excluded      = round(100 * mean(chart$ma_only_excluded), 1),
    pct_immortal_time_risk    = round(100 * mean(chart$time_zero %chin% c("diagnosis_date", "procedure_date")), 1),
    pct_exposure_not_reported = round(100 * mean(chart$exposure_def == "not_reported"), 1)
  )
}

/* ---- PRISMA-ScR flow counts ---- */
/* A record is screened-in at a stage only if NO reviewer excluded it (conservative dual-screen rule). */
proc sql;
  /* Identified and duplicates: dedup on DOI only; DOI-less rows are never duplicates. */
  create table _flow as
  select count(*) as identified,
         calculated identified
           - ( (select count(distinct doi) from work.records where doi ne '')
               + (select count(*)          from work.records where doi  = '') ) as duplicates_removed
  from work.records;

  /* Per-record decision at each stage: included only if min(decision)='include' across reviewers. */
  create table _ta as
  select record_id, max(decision='exclude') as any_excl
  from work.screen where stage='title_abstract' group by record_id;
  create table _ft as
  select record_id, max(decision='exclude') as any_excl
  from work.screen where stage='full_text'      group by record_id;

  create table prisma_scr_flow as
  select f.identified, f.duplicates_removed,
         (select count(*) from _ta)                  as screened_title_abstract,
         (select sum(any_excl) from _ta)             as excluded_title_abstract,
         (select count(*) from _ft)                  as assessed_full_text,
         (select sum(any_excl) from _ft)             as excluded_full_text,
         (select sum(any_excl=0) from _ft)           as included
  from _flow f;
quit;

/* ---- Evidence map: study count by design x outcome_domain (the scoping deliverable, NOT a pooled effect) ---- */
proc freq data=work.chart noprint;
  tables design*outcome_domain / out=evidence_map(drop=percent);
run;

/* ---- Design-credibility prevalence the map should disclose (without grading any single study) ---- */
proc sql;
  create table credibility_flags as
  select count(*) as n_included,
         round(100*mean(design='prevalent_user'),0.1)                              as pct_prevalent_user,
         round(100*mean(design='active_comparator_new_user'
                        and washout_days ne . and washout_days>=365),0.1)          as pct_acnu_with_washout,
         round(100*mean(ma_only_excluded=1),0.1)                                   as pct_ma_only_excluded,
         round(100*mean(time_zero in ('diagnosis_date','procedure_date')),0.1)     as pct_immortal_time_risk,
         round(100*mean(exposure_def='not_reported'),0.1)                          as pct_exposure_not_reported
  from work.chart;
quit;

Citations

1. [1]Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005;8(1):19-32.
2. [2]Tricco AC, Lillie E, Zarin W, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Annals of Internal Medicine. 2018;169(7):467-473.
3. [3]Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implementation Science. 2010;5:69.
4. [4]Munn Z, Peters MDJ, Stern C, et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology. 2018;18:143.