JBI Critical Appraisal Checklist for Prevalence Studies

What it is

— The JBI Critical Appraisal Checklist for Prevalence Studies is a nine-item critical-appraisal (risk-of-bias) instrument maintained by JBI (formerly the Joanna Briggs Institute, Adelaide) as part of its suite of design-specific appraisal tools and the JBI Manual for Evidence Synthesis. It was developed by Munn and colleagues specifically because generic observational-study appraisal tools (e.g., the Newcastle-Ottawa Scale, STROBE-derived checklists) do not interrogate the things that actually threaten a prevalence estimate: how the sample was drawn, whether it represents the target population, how the condition was measured, and whether the proportion and its precision were reported correctly. The nine items ask whether (1) the sample frame appropriately addressed the target population, (2) study participants were sampled appropriately, (3) the sample size was adequate, (4) study subjects and setting were described in detail, (5) data analysis covered the identified sample with sufficient coverage, (6) valid methods identified the condition, (7) the condition was measured in a standard, reliable way for all participants, (8) appropriate statistical analysis was used, and (9) the response rate was adequate (or low response rate was managed appropriately). It is a tool for appraising included studies inside a systematic review of prevalence, not a reporting checklist for authors and not a numeric quality score.

When to use

— Use this checklist when you are conducting (or reviewing) a systematic review or meta-analysis of prevalence, point-prevalence, period-prevalence, or cumulative-incidence (risk) estimates and need to assess the methodological quality / risk of bias of each included cross-sectional or descriptive study. Typical decision contexts: an HTA/payer epidemiology section that must justify the population size or eligible-patient counts behind a budget-impact or cost-effectiveness model; a burden-of-disease or unmet-need chapter in a value dossier; a peer-reviewed prevalence systematic review; or a regulatory background-epidemiology section supporting an orphan-designation or natural-history submission. Decision rules for choosing THIS tool over siblings: if your review question is "how common is the condition?" use JBI Prevalence; if it is "does exposure A cause/associate with outcome B?" use a tool for analytic observational studies (JBI for cohort/case-control, or ROBINS-I / Newcastle-Ottawa); if it is "how accurate is a diagnostic test?" use QUADAS-2 / JBI diagnostic-test-accuracy; if you are authoring and need a reporting checklist (not appraisal) reach for STROBE (and, for routinely-collected health data, RECORD), which are complementary to — not substitutes for — this appraisal tool.

What it requires

— Translated into real-world-data terms, the nine items enforce four substantive domains. (1) Sampling and target-population fit (items 1-3, 9): the source population and sampling frame must map to the population the estimate is meant to describe, with adequate size and an adequate, non-differential response/capture rate — in claims/EHR this becomes whether the enrolled or empaneled population is representative of the target and whether denominators are correctly defined. (2) Setting and case definition transparency (items 4, 6): the population, setting, and condition definition must be described in enough detail to reproduce — in RWD this is precisely phenotype/algorithm specification (code lists, diagnosis-window logic) and documentation of the data source. (3) Coverage and measurement validity/reliability (items 5, 7): the analysis must cover the identified sample with low attrition and the condition must be measured the same validated way in everyone — in RWD this is algorithm validation (PPV/sensitivity), continuous-enrollment requirements, and consistent ascertainment across subgroups. (4) Correct statistical reporting (item 8): the proportion must be reported with an appropriate confidence interval and, where relevant, stratification — not a point estimate alone. Note what the tool deliberately does not assess: it is not designed for confounding control, time-zero alignment, estimands/intercurrent events, or comparative effect estimation, because prevalence is a descriptive parameter, not a causal contrast.

When NOT to use — limitations and common misapplications

— This is a risk-of-bias appraisal instrument, not a reporting guideline and not a validated quality score: JBI itself discourages summing the nine items into a numeric cut-off, because an arbitrary "7/9 = high quality" threshold hides which domain failed and weights non-equivalent items equally. Do not use it to appraise analytic/comparative designs — applying a prevalence tool to a cohort study that estimates a hazard ratio leaves confounding, immortal time, and selection entirely unexamined; that is a wrong-tool error directly analogous to using STROBE where RECORD-PE is required. Do not treat completion of the checklist as evidence the estimate is unbiased or generalizable — a study can pass all nine items within a non-representative claims population and still produce a prevalence that does not transport to the decision population. Avoid checklist-as-theater: marking "yes/no/unclear" without extracting the underlying sampling frame, denominator, and algorithm validity adds no information. Finally, a single JBI per-study appraisal does not substitute for a certainty-of-evidence judgment across the body of prevalence estimates (e.g., GRADE-style downgrading for heterogeneity and indirectness).

How it maps to this catalog

— Several catalog concepts implement what individual JBI items demand for claims/EHR/registry reviews. Item 6 (valid identification of the condition) and item 4 (case-definition detail) are implemented by diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe, which gives the operational code-list and time-window logic, and its validity is what item 7 (standard, reliable measurement) actually scores. Item 5 (adequate coverage of the identified sample) and item 9 (response/capture rate) are implemented by attrition-and-loss-to-follow-up-rwe, which formalizes continuous-enrollment and loss-to-capture handling. Items 1-3 (sample frame and target-population fit) connect to claims-analysis for denominator construction and to data-source representativeness work (e.g., Medicare FFS vs Medicare Advantage vs commercial coverage differences) that determines whether the numerator/denominator describe the intended population. When a downstream review pivots from descriptive prevalence to a comparative or causal question, the relevant concepts shift to active-comparator-new-user, high-dimensional-propensity-score-hdps-rwe, estimands-ate-att-intercurrent-events-rwe, and target-trial-emulation — a signal that JBI Prevalence is no longer the right appraisal instrument. Applied note for RWD: when appraising a claims-based prevalence study, read item 1 as "is the enrolled population the target population?", item 5/9 as "is the continuous-enrollment denominator complete and non-differential?", and item 6/7 as "is the phenotype validated (reported PPV/sensitivity) and applied identically to all enrollees?" — these three questions carry most of the bias in real-world prevalence estimates.