PROBAST+AI
An updated tool for evaluating quality, risk of bias, and applicability in prediction-model studies using regression or artificial-intelligence methods; it is not limited to AI models.
On this page
What this is
An updated tool for evaluating quality, risk of bias, and applicability in prediction-model studies using regression or artificial-intelligence methods; it is not limited to AI models. PROBAST+AI distinguishes quality of model development from risk of bias in performance evaluation and applicability to the intended use. Record the model, outcome, population, evaluation component, and tool version before answering questions. The official paper and resources supply the exact instrument; this entry provides a catalog crosswalk.
When to use
Use for appraisal of studies developing or evaluating clinical prediction models, including internal and external validation components as relevant. Pair it with appropriate TRIPOD reporting guidance and operational prediction-validation concepts. The unit of appraisal and performance estimate must be explicit when one paper reports several models or datasets.
What it requires and practical domains
The analyst or appraiser should keep the question, population, outcome or model result, source documents, version, and judgments connected. Use the domains below to organize that evidence. They are a condensed implementation crosswalk; consult the linked official source for exact instrument text, permissions, and decision rules. Retain disagreements and missing information rather than converting them into favorable answers.
When NOT to use and limitations
Do not apply it as a causal-effect risk-of-bias tool, assign a numeric total as a validated quality score, assume a high AUC establishes low bias, or infer deployment safety from complete reporting or a favorable appraisal alone. A completed form documents an assessment; it does not create missing study evidence, guarantee future performance, or authorize a clinical, regulatory, or reimbursement conclusion. Applicability remains a separate question from internal validity and reporting completeness.
How it maps to this catalog
Follow the related concepts to implement definitions, analysis, diagnostics, and interpretation. For this entry the companion routes are prediction-model-validation-recalibration-rwe, cross-validation-and-overfitting-rwe, brier-score-calibration-rwe, tripod, probast. Preserve the distinction between the guidance/tool and the methods that implement it; source-backed concepts explain those methods, while this resource organizes reporting or appraisal evidence.
Evidence record
For each applicable domain record the exact evidence location, relevant study/model version, judgment and rationale, uncertainty, unresolved issue, responsible reviewer, and review date. Do not invent approvals, validation results, or completed analyses to fill a field. Store the actual operational specification separately from the assessment, so a favorable label cannot silently replace the underlying evidence.
Checklist
- Identify the model, intended use, population, outcome, and performance estimate being appraised.
- Keep development-quality judgments separate from evaluation-bias judgments and applicability.
- Assess data provenance, predictor/outcome availability and measurement, sample information, and analysis.
- Inspect leakage, preprocessing, tuning, resampling, missingness, calibration, and external evaluation.
- Document signaling answers, judgment rationale, evidence locations, and unresolved information.