ROBINS-E (Risk Of Bias In Non-randomized Studies - of Exposures)

What it is

ROBINS-E (Risk Of Bias In Non-randomized Studies - of Exposures) is a domain-based, signalling-question critical-appraisal tool that assesses the risk of bias in the effect estimate reported by a non-randomized follow-up (cohort) study of an exposure. It is the exposures sibling of ROBINS-I (interventions): it inherits ROBINS-I's "emulated target trial" logic — judge the observational study against the hypothetical randomized experiment it is trying to approximate — but rewrites the domains and signalling questions for the realities of exposure research, where the "exposure" is an environmental, occupational, dietary, or other non-prescribed agent that no one assigned. It is maintained by the ROBINS-E Development Group (an international collaboration including the ROBINS-I authors and environmental-health methodologists, hosted at riskofbias.info) and was developed under the program to adapt GRADE for environmental health; the launch tool was described by Higgins, Morgan, Rooney, Taylor, Thayer and colleagues in Environment International (2024). ROBINS-E is a risk-of-bias instrument, not a reporting checklist and not a quality score: its output is a per-domain and overall judgment (Low risk / Some concerns / High risk / Very high risk) for a specified result, structured around a clearly stated PECO (Population, Exposure, Comparator, Outcome) question.

When to use

Apply ROBINS-E when you are appraising, in a systematic review or evidence synthesis, a non-randomized cohort/follow-up study estimating the causal effect of an exposure on a health outcome, and you need a transparent, reproducible bias assessment to feed a GRADE certainty-of-evidence rating. Its decision context is environmental and occupational health risk assessment (e.g., EPA/IRIS, NTP, IARC monographs, EFSA opinions), nutritional epidemiology reviews, and Cochrane-style reviews of exposure questions, plus the peer-reviewed reviews that underpin them. The governing decision rule for choosing the right sibling instrument: if the "exposure" is a therapeutic intervention (a drug, device, procedure, or program someone decided to give), appraise it with ROBINS-I, not ROBINS-E. If it is a non-assigned exposure (air pollution, PFAS, silica dust, a dietary pattern, a behavior), ROBINS-E is the tool. ROBINS-E targets follow-up designs; it is not built for case-control or cross-sectional designs, which fall outside its current scope and require other appraisal approaches.

What it requires

ROBINS-E first fixes a PECO and a specific numerical result to be appraised (you assess a result, not a study), then works through seven bias domains, each driven by signalling questions answered Yes / Probably yes / Probably no / No / No information:

1. bias due to confounding — were the important confounders of the exposure-outcome relationship identified and adequately controlled, given that exposure was not randomized;
2. bias in measurement of the exposure — was exposure assessed validly and reliably, and was assessment differential with respect to the outcome (recall, exposure-misclassification, and the validity of the exposure metric);
3. bias in selection of participants into the study — selection related jointly to exposure and outcome, including selection at or after the start of follow-up;
4. bias due to post-exposure interventions — actions taken after exposure that differ by exposure level and affect the outcome;
5. bias due to missing data — missingness in exposure, outcome, or confounders and whether it could distort the estimate;
6. bias in measurement of the outcome — outcome ascertainment validity and whether it was differential by exposure; and
7. bias in selection of the reported result — selective reporting from multiple measurements, analyses, or subgroups. Each domain rolls up to a domain-level judgment, and the domains combine (worst-domain- dominant logic, with a distinct "Very high" tier) into an overall risk-of-bias rating for that result. The tool also asks the assessor to record the predicted direction of each bias, which feeds the downstream GRADE judgment.

When NOT to use — limitations and common misapplications

1. Wrong sibling instrument (the dominant error). Using ROBINS-E to appraise a drug/intervention cohort — e.g., a claims- based active-comparator new-user study of two antidiabetics — is a category error; that is the ROBINS-I lane. Conversely, forcing ROBINS-I onto an environmental exposure study misframes the confounding and exposure-measurement domains.
2. Wrong design. ROBINS-E is for follow-up studies; applying it to case-control or cross-sectional studies stretches it past its validated scope.
3. A risk-of-bias tool is not a reporting checklist — a study can be beautifully reported (STROBE/RECORD-compliant) and still be High risk in ROBINS-E, and vice versa; do not substitute one for the other.
4. It is not a numeric quality score. ROBINS-E deliberately avoids summing items into a scale; converting domain ratings into points and averaging them discards the worst-domain logic the tool is built on.
5. Result-level, not study-level. A single paper can yield Low risk for one outcome and High risk for another; assessing "the study" rather than a defined PECO result is a misuse.
6. It does not manufacture causality — rating a study Low risk does not make an observational association causal; it only certifies that internal bias is judged low for that estimate.
7. Known critiques (Bero et al., 2018, raised during development) flag that early versions were difficult to apply consistently and risked over-penalizing or under-penalizing confounding in observational exposure science; assessor training, pilot calibration, and dual independent assessment with reconciliation are necessary to get reproducible ratings.
8. Checklist-as-theater — answering signalling questions without the underlying methodological judgment (e.g., waving through "confounding adequately controlled" without scrutinizing the confounder set) defeats the instrument.

How it maps to this catalog

ROBINS-E's seven domains are appraisal lenses; in this repo the underlying methods a study must execute well to earn a Low-risk rating are implemented by concepts the assessor can check against, domain by domain:

• Confounding (Domain 1): the study should demonstrate principled confounder selection and residual-confounding accounting — unmeasured-confounding-probabilistic-bias-analysis-rwe, e-value-sensitivity-analysis, negative-control-outcomes-rwe / negative-control-exposures-rwe, and (where a propensity approach is used) propensity-score-methods-psm-iptw; the estimand being targeted should be explicit via estimands-ate-att-intercurrent-events-rwe.
• Exposure measurement (Domain 2): validity of the exposure metric / phenotype maps to algorithm-validation (and claims-outcome-algorithm-ppv-sensitivity-rwe when an administrative-data exposure proxy is used).
• Participant selection (Domain 3): selection-bias-sensitivity-analysis-rwe, time-zero-index-date-alignment-rwe (selection/immortal-time at the start of follow-up), and immortal-time-bias-handling.
• Missing data (Domain 5): missing-data-pattern-table-rwe and attrition-and-loss-to-follow-up-rwe.
• Outcome measurement (Domain 6): algorithm-validation again, for outcome-ascertainment validity and whether it is differential by exposure.
• Overall / synthesis: the magnitude and direction of residual bias the tool asks you to record are quantified with the quantitative-bias-analysis-toolkit-rwe, and external validity of the appraised estimate (a GRADE indirectness concern downstream) with generalizability-transportability-external-validity-rwe. Note these are ROBINS-I-lane pharmacoepi designs only by analogy; active-comparator-new-user, high-dimensional-propensity-score-hdps-rwe, and claims-analysis belong to the intervention sibling and are not the natural exemplars for ROBINS-E.

Applied note (when an administrative-data cohort meets ROBINS-E)

ROBINS-E's home substrate is environmental/occupational/nutritional cohorts (e.g., a PFAS-serum cohort and kidney cancer, an ambient-PM2.5 cohort and cardiovascular mortality, a dietary-pattern cohort and incident diabetes), where exposure measurement and confounding are the dominant biases. Claims/EHR data can host a ROBINS-E-appropriate study only when the cohort studies a non-intervention exposure captured in those data — for example, an occupational or environmental exposure recorded in linked records — in which case the exposure-measurement domain hinges on how well the administrative proxy validates against true exposure (algorithm-validation), and the confounding domain hinges on the completeness of the recorded confounder set. The moment the "exposure" is a prescribed therapy, switch to ROBINS-I.

Citations

1. [1]Higgins JPT, Morgan RL, Rooney AA, Taylor KW, Thayer KA, Silva RA, et al. A tool to assess risk of bias in non-randomized follow-up studies of exposure effects (ROBINS-E). Environment International. 2024;186:108602.
2. [2]Morgan RL, Whaley P, Thayer KA, Schunemann HJ. Identifying the PECO: a framework for formulating good questions to explore the association of environmental and other exposures with health outcomes. Environment International. 2018;121(Pt 1):1027-1031.
3. [3]Bero L, Chartres N, Diong J, Fabbri A, Ghersi D, Lam J, et al. The risk of bias in observational studies of exposures (ROBINS-E) tool: concerns arising from application to observational studies of exposures. Systematic Reviews. 2018;7(1):242.

1. [4]ROBINS-E Development Group. ROBINS-E: Risk Of Bias In Non-randomized Studies - of Exposures. riskofbias.info (maintained tool, guidance documents, and detailed templates).