JBI Critical Appraisal Tool for Cohort Studies

What it is

The JBI Critical Appraisal Tool for Cohort Studies is a structured instrument for judging the internal validity (risk of bias) of cohort studies during systematic reviews and evidence synthesis. It is developed and maintained by JBI (formerly the Joanna Briggs Institute, University of Adelaide), the organization behind the JBI suite of design-specific appraisal tools (cohort, case-control, RCT, prevalence, case series, analytical cross-sectional, etc.) used across JBI- and Cochrane-adjacent reviews. The original tool was an 11-item checklist scored Yes / No / Unclear / Not applicable. The 2024 revision (Barker et al.) is a substantive redesign: items were rewritten, mapped to explicit bias domains (group comparability and selection, exposure measurement, confounding identification and management, outcome measurement, follow-up completeness and attrition, and appropriateness of the statistical analysis), and — critically — the tool was reframed as a per-domain risk-of-bias judgment that converges toward an overall judgment, NOT a numeric quality score. It is an appraisal tool, not a reporting checklist.

When to use

Use the JBI Cohort tool when you are the reviewer/appraiser of a cohort study within an evidence synthesis (systematic review, scoping-to-synthesis, rapid review, or evidence brief) that feeds a peer-reviewed publication or an HTA / payer dossier. It is the natural appraisal instrument when your review protocol already follows JBI methodology for reviews of etiology/association or effectiveness. Decision rules versus siblings: appraise a cohort (including most non-interventional RWE cohorts built from claims/EHR/registry) with this tool; switch to the JBI Case-Control tool for case-control designs, the JBI Analytical Cross-Sectional tool for prevalence/ cross-sectional designs, and the JBI RCT tool for randomized trials. For target-trial-emulation studies that mimic an RCT but are analyzed as cohorts, the cohort tool is appropriate, supplemented by emulation-specific scrutiny. ROBINS-I is a reasonable alternative when the review demands a per-outcome, signed-bias framework tied to a target trial; many HTA bodies accept either, and some prefer ROBINS-I for intervention questions. JBI Cohort is for appraising a study, not for reporting your own — that is the job of STROBE (general observational), RECORD / RECORD-PE (routinely collected health data and pharmacoepidemiology), or HARPER (HARmonized Protocol Template for pharmacoepi).

What it requires

The tool forces a reviewer to interrogate the bias-relevant features of the study, framed here for real-world data: - Comparability and selection of groups — were exposed and unexposed (or comparator) groups recruited from the same source population and comparable on baseline prognosis? In RWE this is the active-comparator / new-user question. - Exposure measurement — was exposure measured validly and the same way in both groups? For claims/EHR this is algorithm/phenotype validity (e.g., dispensing-based exposure, time-zero alignment). - Confounding — were the important confounders identified, and were strategies stated to deal with them (restriction, matching, propensity scores, regression)? The revision separates identification from management. - Outcome measurement — were outcomes measured validly, reliably, and blind to exposure where relevant? In RWE this maps to outcome-algorithm performance (PPV/sensitivity). - Follow-up and attrition — was follow-up long enough and complete, and was incomplete follow-up addressed? - Statistical analysis — was the analysis appropriate (including handling of time, confounding, and missing data)? The reviewer reaches a domain-level risk-of-bias judgment (e.g., low / moderate / high / unclear) and an overall judgment supported by recorded rationale — not a tally of "Yes" answers.

When NOT to use — limitations and common misapplications

- It is not a reporting guideline. Completing JBI Cohort tells you whether a study is at risk of bias, not whether it is well reported. Using it where STROBE / RECORD-PE / HARPER is required (e.g., a journal or regulator asking for reporting completeness) is a category error. - It is not a quality score. The single most common misapplication — endemic with the older 11-item version — is summing the "Yes" answers into a numeric quality score and dichotomizing studies at an arbitrary cut-point (e.g., "≥7/11 = high quality"). The 2024 revision was designed precisely to stop this: item counts are not interchangeable, a single fatally biased domain can invalidate a study with many "Yes" items, and meta-analytic weighting by such scores is methodologically indefensible. Report domain judgments, not a score. - Appraisal is not causal inference. A "low risk of bias" rating does not make an observational study causal; it means the reported design and analysis guard against known biases. Unmeasured confounding can remain invisible to any checklist. - Checklist-as-theater. Marking items without recording the specific evidence and rationale (the study text, the algorithm, the attrition figures) produces a defensible-looking artifact with no analytic content. HTA reviewers discount unjustified ratings. - Wrong tool for the design. Applying the cohort tool to a case-control or cross-sectional study (or vice versa) mis-frames the relevant biases.

How it maps to this catalog

Each JBI domain is implemented by one or more concepts here: - Group comparability / selection → active-comparator-new-user, target-trial-emulation, selection-bias-sensitivity-analysis-rwe. - Exposure measurement / time-zero → diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe, immortal-time-bias-handling. - Confounding identification and management → high-dimensional-propensity-score-hdps-rwe, propensity-score-methods-psm-iptw, unmeasured-confounding-probabilistic-bias-analysis-rwe, negative-control-outcomes-rwe, estimands-ate-att-intercurrent-events-rwe. - Outcome measurement → claims-outcome-algorithm-ppv-sensitivity-rwe. - Follow-up / attrition → attrition-and-loss-to-follow-up-rwe. - Data substrate / overall feasibility → claims-analysis.

Applied note (claims/EHR/registry RWE)

When appraising a claims- or EHR-based cohort, do not accept a face-value "exposure was measured" or "outcomes were ascertained." Demand the operational evidence behind each domain: the washout and continuous-enrollment definition behind incident-user status; the phenotype/algorithm and its validation (PPV, sensitivity) behind exposure and outcomes; the covariate-measurement window and confounding-control strategy; the attrition funnel and how disenrollment, death, and switching were handled; and the time-zero alignment that prevents immortal time. The JBI domains give you the questions; the catalog concepts give you the standard against which a "low risk of bias" judgment can actually be defended in an HTA dossier.