ROBINS-I (Risk Of Bias In Non-randomised Studies of Interventions)

What it is

— ROBINS-I (Risk Of Bias In Non-randomised Studies of Interventions) is a structured, domain-based tool for the critical appraisal of an individual non-randomized study estimating the effect of an intervention. It was introduced by Sterne, Hernán, Higgins, Reeves and colleagues (BMJ, 2016) and is developed and maintained by the ROBINS-I development group and the Cochrane Bias Methods Group, distributed at riskofbias.info — it is not an EQUATOR reporting guideline. Its defining feature, which separates it from every reporting checklist, is the target-trial framing: the assessor first specifies the hypothetical pragmatic randomized trial whose effect the observational study is trying to estimate, then judges bias as deviation of the actual study from that target trial. Appraisal proceeds through signaling questions in seven bias domains — (1) confounding, (2) selection of participants into the study, (3) classification of interventions, (4) deviations from intended interventions, (5) missing data, (6) measurement of outcomes, and (7) selection of the reported result — yielding a judgment per domain and an overall judgment on a five-level qualitative scale: Low, Moderate, Serious, Critical, or No information. It is a risk-of-bias instrument, in the same family as RoB 2 (for randomized trials) and ROBINS-E (for exposure/etiologic studies), and is the appraisal counterpart to — not a substitute for — reporting checklists (STROBE, RECORD-PE), protocol templates (HARPER, ENCePP), and systematic-review appraisal tools (AMSTAR 2).

When to use

— Use ROBINS-I to appraise the internal validity of a primary non-randomized study of an intervention — typically a comparative cohort study of two treatment strategies in claims, EHR, registry, or linked data, including target-trial emulations and active-comparator new-user cohorts. It is the standard within-study risk-of-bias tool for the included non-randomized studies of a Cochrane or other systematic review, and it feeds directly into GRADE certainty rating for a body of non-randomized evidence (Schünemann et al., 2019). It is increasingly expected in HTA/payer evidence assessments and in regulatory contexts (FDA/EMA) where the credibility of an observational comparative-effectiveness or safety estimate must be defended. Decision rule for choosing the right sibling: appraise a randomized trial with RoB 2, not ROBINS-I; appraise an exposure/etiologic study with no defined intervention (e.g., effect of an environmental or behavioral exposure on harm) with ROBINS-E, not ROBINS-I; appraise the systematic review itself with AMSTAR 2; and report the primary study with STROBE/RECORD-PE. ROBINS-I is for an interventional contrast you can cast as a target trial.

What it requires

— Substantively, ROBINS-I forces the appraiser to make explicit the things that determine whether an observational estimate is credible. The target-trial step demands a stated PICO/estimand (eligibility, the intervention strategies compared, the comparator, the outcome, and the effect of interest — assignment vs adherence). The confounding domain requires a pre-specified list of important confounders and time-varying confounders affected by prior treatment, and an appraisal of whether the analysis controlled them appropriately (e.g., propensity-score or high-dimensional adjustment, g-methods) — this is the domain that most often drives a Serious or Critical rating in real-world data. The selection domain interrogates time-zero alignment and immortal-time bias (was selection and start of follow-up tied to the intervention decision?). The classification-of-interventions domain asks whether exposure status was defined and ascertained without knowledge of the outcome — central where exposure is an algorithm over dispensing or administration records. The deviations domain addresses whether the analysis targets the intended assignment-vs-adherence estimand and handles switching/discontinuation (intercurrent events). The missing-data domain addresses attrition, loss to follow-up, and informative censoring. The outcome-measurement domain asks whether the outcome phenotype is valid and ascertained blind to exposure. The reported-result domain addresses selective reporting against a pre-specified analysis plan. For claims/EHR/registry RWE these map onto data-fitness-for-use, phenotype/algorithm validation (PPV/sensitivity), and sensitivity/quantitative bias analysis.

When NOT to use — limitations and common misapplications

— ROBINS-I is a risk-of-bias tool, not a reporting checklist, not a study-quality scale, and not a guarantee of validity. Concrete failure modes: (1) Wrong tool for the design — applying ROBINS-I to a randomized trial (use RoB 2) or to a non-interventional exposure/etiologic study with no intervention contrast (use ROBINS-E). (2) Summing or scoring domains — ROBINS-I yields qualitative per-domain and overall judgments; adding them into a points total or a "quality percentage" is a misuse the developers explicitly reject, and a single Critical domain caps the overall rating at Critical regardless of the others. (3) Skipping the target-trial step — answering signaling questions without first specifying the hypothetical trial and the estimand produces incoherent, non-reproducible ratings; the target trial is the instrument. (4) Confusing appraisal with reporting — completing ROBINS-I does not make a study well-reported (that is STROBE/RECORD-PE) and, crucially, completing ROBINS-I does not make an observational estimate causal or unconfounded — a Low overall rating reflects the appraiser's judgment given measured confounders, not proof of exchangeability. (5) Confusing it with AMSTAR 2 — ROBINS-I appraises the included primary studies; AMSTAR 2 appraises the review. (6) Checklist-as-theater — recording domain judgments without documenting the rationale and the confounder list defeats the auditability the tool exists to create. Note that ROBINS-I V2 is in development, refining the signaling questions and domain structure; cite the version actually applied.

How it maps to this catalog

— ROBINS-I tells the reviewer what must be controlled; the catalog concepts implement how. Domain-by-domain crosswalk to implementing concepts in this repo: - Target-trial / estimand framing (the prerequisite step) → target-trial-emulation and estimands-ate-att-intercurrent-events-rwe specify the hypothetical trial, the estimand, and the handling of intercurrent events the appraisal judges against. - Confounding domain → high-dimensional-propensity-score-hdps-rwe and active-comparator-new-user implement the confounder-control and channeling-mitigation that move a study away from a Serious/Critical confounding rating. - Selection / time-zero domain → active-comparator-new-user (time-zero at initiation, no immortal time) and target-trial-emulation (aligned eligibility, assignment, and start of follow-up). - Classification-of-interventions & outcome-measurement domains → diagnosis-phenotype-algorithm-1ip-2op-time-window-rwe supplies the validated exposure/outcome algorithms (1 IP / 2 OP rules, time windows, PPV) the appraisal requires. - Missing-data domain → attrition-and-loss-to-follow-up-rwe implements attrition accounting and informative-censoring handling. - Data fitness underlying every domain → fit-for-purpose-data-assessment-rwe and claims-analysis establish whether the source data can support the operational definitions at all.

Applied note (claims/EHR/registry RWE)

When appraising a claims- or EHR-based comparative study, resolve the target trial first (eligibility, the two treatment strategies, time zero, the estimand), then attack confounding and selection: confirm an active-comparator new-user design with time-zero at initiation, a high-dimensional propensity score over the lookback window, and that "no prior fill" reflects observed continuous enrollment rather than Medicare Advantage missingness. For the classification and outcome domains, demand the phenotype's validation metrics (PPV/sensitivity) and ascertainment blind to exposure. For missing data, require a CONSORT-style attrition flow and treatment of loss to follow-up as potentially informative, with quantitative bias analysis (e.g., E-value, negative controls) supporting the reported-result domain. A clean ROBINS-I appraisal documents the confounder list and the rationale for each domain judgment, not just the five-level labels.