DAGs and the Backdoor Criterion for Drug Studies

In plain language

A DAG (directed acyclic graph) is a simple diagram where you draw every variable in your study as a box and draw arrows showing which variables cause which — it is a map of your causal assumptions, not a picture of your data. The backdoor criterion is a rule you apply to that map: it tells you exactly which variables you must adjust for to make a fair comparison between drug users and non-users, and — just as importantly — which variables you must leave alone because adjusting for them would actually introduce bias. The core insight is that some variables (confounders) open unfair comparison paths that you need to close by adjusting, while other variables (colliders) are naturally blocking a spurious path and will blow it open the moment you condition on them.

A directed acyclic graph (DAG) is a graph of nodes (variables) and directed edges (assumed direct causes) with no directed cycles. In pharmacoepidemiology and RWE it is the tool that turns informal clinical reasoning about confounding, selection, and timing into an explicit, falsifiable model that dictates what to adjust for before any propensity score, regression, or g-method is fit. The DAG does not estimate an effect; it tells you which adjustment set licenses an unbiased estimate under the drawn assumptions, and — just as importantly — which variables would introduce bias if conditioned on.

Core conceptual distinction

The backdoor criterion (Pearl) states that a set Z is sufficient to identify the effect of exposure A on outcome Y if (1) Z blocks every "backdoor" path — every path from A to Y that begins with an arrow into A — and (2) Z contains no descendant of A. Three structures behave differently and must not be confused. A confounder (a common cause of A and Y, e.g. disease severity) opens a backdoor path that you must close by conditioning. A mediator (a variable on the causal path A → M → Y, e.g. post-initiation adherence or on-treatment LDL) lies on the front-door path; conditioning on it removes part of the very effect you are estimating (total-effect bias) and can open a collider path. A collider (a common effect of two variables, e.g. database inclusion caused by both healthy-user behavior and the outcome) is naturally path-blocking; conditioning on it (or on its descendant) opens a spurious path and induces collider-stratification / selection bias. The estimand must be fixed first: a DAG that identifies a total effect (adjust confounders only) is different from one identifying a controlled direct effect (which legitimately conditions on a mediator and then needs mediator-outcome confounders too). M-bias is the canonical trap — conditioning on a pre-exposure variable that is itself a collider of two unmeasured causes can create bias where none existed.

Pros, cons, and trade-offs

- vs ad hoc "adjust for everything available" / change-in-estimate covariate screening: A DAG-derived adjustment set is principled and avoids over-adjustment. The "throw in every baseline claims variable" habit and stepwise/change-in-estimate selection routinely include mediators and colliders, amplifying rather than removing bias (Schisterman 2009). Cost: the DAG is only as good as its assumptions, which are usually unverifiable; it offloads the hard problem onto subject-matter judgment. Prefer the DAG whenever the candidate covariate list contains anything measured after, or plausibly affected by, the exposure decision. - vs disjunctive-cause / pre-exposure-covariate heuristics (VanderWeele): Practical heuristics ("adjust for any pre-exposure cause of exposure or outcome") are robust defaults when a full DAG is infeasible and avoid M-bias in most realistic settings. Cost: they can over-adjust relative to a minimal sufficient set and give no guidance on selection/collider structure or time-varying feedback. Prefer an explicit DAG for selection bias, time-varying confounding, and when efficiency (minimal set) matters. - vs propensity-score / IPTW methods directly: The DAG and the PS are complementary, not competing — the DAG decides which variables enter the PS; the PS decides how to balance them. A perfectly balanced PS on the wrong variable set (e.g. a post-baseline mediator) is confidently biased. Always draw the DAG first, then estimate.

When to use

Draw the DAG at the protocol/SAP stage of every comparative drug, procedure, or policy study, before specifying the covariate set, the PS model, or the g-method. It is most valuable when (a) the candidate covariate list mixes pre- and post-index variables; (b) selection into the cohort is non-trivial (continuous-enrollment, complete-case, or landmark restrictions); (c) treatment-confounder feedback is present and you must decide between ordinary regression/PS and g-methods (MSMs, g-formula); or (d) you must justify to a regulator or HTA body why a given variable was or was not adjusted.

When NOT to use — and when it is actively misleading or dangerous

- As a substitute for data. A DAG asserting "no unmeasured confounding" does not make it so. In claims, severity and frailty are usually unmeasured; the DAG should show them as unadjusted common causes and trigger negative-control or quantitative-bias analysis, not paper over them. - When time ordering is wrong or implicit. The single most common fatal DAG error in RWE is treating a post-index variable (adherence, on-treatment biomarker, subsequent utilization) as if it were baseline. If a node's timestamp relative to time zero is ambiguous, the DAG is dangerous: it will license adjusting for a mediator or collider. Every node needs an explicit "measured at/before/after time zero" label. - When the analyst conditions on a collider to "increase precision." Adjusting for a strong predictor of the outcome that is a descendant of exposure (or a common effect of exposure and an outcome cause) can look like it tightens the estimate while biasing it. Over-adjustment is not conservative. - When a hand-drawn DAG is over-trusted. A sparse DAG that omits a real common cause silently asserts its absence. The graph encodes your assumptions; it cannot detect the arrow you forgot to draw. Use implied conditional-independence tests against the data to falsify (not confirm) the structure.

Data-source operational depth

- Claims (FFS or commercial): Severity, frailty, performance status, and prescriber preference are the dominant confounders and are mostly unmeasured; draw them explicitly as latent common causes and rely on high-dimensional proxies (prior diagnoses, procedures, drug classes, utilization counts in the lookback). Failure modes: (1) MA-only person-time lacks fee-for-service claims, so a "no prior diagnosis/fill" proxy is missingness, not absence — exclude MA-only spans or the unmeasured-confounding node becomes a selection node too. (2) Healthcare-utilization intensity is a collider/proxy hybrid — it is caused by severity and by access, and outcomes are only observed in those who interact with the system; adjusting for total post-index utilization conditions on a descendant of exposure. (3) Continuous-enrollment requirements after index condition on survival/observability, a collider of exposure and outcome (informative censoring). (4) Differential competing risks (e.g. death in elderly claims cohorts) act as a censoring node that differs by arm and must appear in the DAG. - EHR: Labs and vitals sharpen severity (an advantage over claims) but their presence is driven by the visit/ordering process — the DAG must include the observation/measurement process node, or conditioning on "had a baseline HbA1c" silently conditions on care-seeking. Loss to follow-up when a patient leaves the network is potentially informative censoring (a collider on outcome and exposure-related health). - Registry: Clinical stage and biomarkers improve exchangeability but are often recorded after the treatment decision; if a stage variable is post-decision it may be a mediator, not a confounder — the timestamp determines the edge direction. - Linked claims–EHR–vital records: Best severity + completeness + mortality, but linkage itself is a selection node (only the linkable subset), and order/fill/service-date discrepancies must be reconciled so every node's position relative to time zero is correct before the backdoor set is read off the graph.

Worked claims example

Question: incident acute kidney injury (AKI) under SGLT2 inhibitor vs DPP-4 inhibitor among adults with type 2 diabetes in a commercial + Medicare FFS database. Nodes and timestamps: baseline (measured in the 365-day lookback ending at the index `fill_date`) — `eGFR_proxy` (CKD diagnosis codes), `diabetes_severity` (insulin use, HbA1c proxies), `prior_utilization`, `prescriber_preference`, plus latent `frailty`/`true_severity`; time zero — `treatment` (arm assigned from the NDC dispensed on the first qualifying fill after a 365-day washout of both classes); post-index — `on_treatment_volume_depletion` (a mediator: SGLT2 → volume depletion → AKI), `adherence`, `subsequent_utilization`; outcome — `AKI`. Reading the backdoor criterion off the DAG: the sufficient adjustment set is {`eGFR_proxy`, `diabetes_severity`, `prior_utilization`, `prescriber_preference`} — all pre-index common causes, all measurable only in [`index_date` − 365, `index_date`] and fed into a high-dimensional propensity score. Variables that must be excluded: `on_treatment_volume_depletion` (mediator of the very effect of interest — adjusting for it estimates a controlled direct effect, not the total effect the safety question wants), `adherence` and `subsequent_utilization` (descendants of treatment → collider/mediator bias), and any post-index continuous-enrollment flag (conditions on observability, a collider of treatment and outcome). Because `true_severity` and `frailty` are unmeasured common causes that remain open backdoor paths, the protocol pre-specifies a negative-control outcome and a quantitative-bias analysis rather than claiming the adjustment set is complete. A DAG is useful only when time ordering is explicit; in RWE virtually every serious DAG error is adjusting for a post-index variable or conditioning on future observability.

Interpreting the output

In the SGLT2 versus DPP-4 study, the DAG analysis produces: sufficient adjustment set = {eGFR_proxy, diabetes_severity, prior_utilization, prescriber_preference}; excluded from adjustment: on_treatment_volume_depletion (mediator), subsequent_utilization (collider descendant).

(1) Formal interpretation. The backdoor criterion confirms that the four pre-index variables block every backdoor path without conditioning on any descendant of the exposure. The mediator on_treatment_volume_depletion lies on the causal path SGLT2 → volume_depletion → AKI; including it in the outcome or propensity model would yield a controlled direct effect, not the total effect the safety question requires. The collider subsequent_utilization receives arrows from both the exposure and an outcome antecedent; conditioning on it opens a spurious non-causal path and induces collider-stratification bias. Two unmeasured nodes — true_severity and frailty — remain as open backdoor paths; the DAG therefore motivates quantitative bias analysis rather than a claim of complete confounding control.

(2) Practical interpretation. The DAG does not produce an effect estimate; it produces a defensible covariate list. A reviewer can trace every variable in the propensity-score specification back to its structural role — confounder (adjust), mediator (exclude), collider (exclude). Variables absent from the model were omitted for explicit causal reasons, not oversight. That audit trail is how a DAG converts clinical judgment into a transparent, contestable analytic choice rather than an ad hoc "include everything in the claims" list.

Worked example

Scenario

A study asks whether Drug A (an SGLT2 inhibitor) reduces the risk of hospitalization compared to Drug B (a DPP-4 inhibitor) in adults with type 2 diabetes. Before fitting any model, the team draws a small DAG with five variables: the drug assignment (Exposure), the hospitalization outcome (Outcome), a baseline disease-severity score (Confounder), a post-treatment lab change caused by Drug A (Mediator), and database enrollment status which is jointly caused by both drug use and the hospitalization outcome (Collider). The question is: which variables should enter the adjustment set, and which should be excluded?

Dataset

Node-and-edge table describing the small study DAG. Each row is one arrow in the diagram. Read 'from -> to' as 'from causes to'.

Steps

• Draw the DAG and list every path from Exposure to Outcome that does NOT go forward along the Exposure → Outcome arrow. The only such path here is: Exposure ← Disease Severity → Outcome. This is the backdoor path — it flows backward into Exposure first, then out to Outcome.

• Apply the backdoor criterion: we need a set of variables that blocks every backdoor path and contains no descendant of Exposure. Disease Severity sits on the one backdoor path and is measured before the drug is prescribed (it is not caused by Exposure), so adjusting for Disease Severity blocks that path. The adjustment set is {Disease Severity}.

• Check the Mediator (Lab Change). Lab Change is caused by Exposure — it is a descendant of Exposure and lies on the causal chain Exposure → Lab Change → Outcome. Adjusting for it would partially erase the drug's effect from the estimate, giving a biased (too-small) result for the total effect. Exclude it.

• Check the Collider (Database Enrollment). Database Enrollment is caused by both Exposure and Outcome. Right now it has two arrows pointing in with no path through it — it naturally blocks any spurious route. The moment you condition on it (say, by requiring everyone to have a follow-up visit), you open a spurious Exposure–Outcome association that did not exist before. Exclude it.

• Read off the final answer: adjust for Disease Severity only. Do not adjust for Lab Change (mediator) or Database Enrollment (collider).

Result

The correct minimal adjustment set is {Disease Severity}. Adjusting for Disease Severity and nothing else closes the one backdoor path (Exposure ← Disease Severity → Outcome) and yields an unbiased estimate of the total effect of Drug A vs Drug B on hospitalization. Adding the Mediator would underestimate the drug's true benefit. Adding the Collider would manufacture a spurious association where none exists.

Runnable example

import networkx as nx
from dowhy import CausalModel
import pandas as pd

# ---- Structural model (timestamps encoded in node names) -------------------------------------
# Pre-index (measured in [index_date-365, index_date]): eGFR_proxy, diabetes_severity,
#   prior_utilization, prescriber_preference. Latent (unmeasured in claims): true_severity, frailty.
# Time zero: treatment. Post-index (descendants of treatment): volume_depletion (mediator),
#   adherence, subsequent_utilization. Outcome: AKI.
edges = [
    ("eGFR_proxy", "treatment"),         ("eGFR_proxy", "AKI"),
    ("diabetes_severity", "treatment"),  ("diabetes_severity", "AKI"),
    ("prior_utilization", "treatment"),  ("prior_utilization", "AKI"),
    ("prescriber_preference", "treatment"),
    ("true_severity", "diabetes_severity"), ("true_severity", "AKI"),   # latent common cause
    ("frailty", "prior_utilization"),       ("frailty", "AKI"),         # latent common cause
    ("treatment", "volume_depletion"),   ("volume_depletion", "AKI"),   # mediator (front-door)
    ("treatment", "adherence"),          ("adherence", "AKI"),          # descendant of treatment
    ("treatment", "subsequent_utilization"),                            # collider feeder
    ("AKI", "subsequent_utilization"),                                  # -> collider
    ("treatment", "AKI"),                                               # effect of interest
]
g = nx.DiGraph(edges)
gml = "graph[directed 1 " + " ".join(
    f'node[id "{n}"]' for n in g.nodes()
) + " " + " ".join(
    f'edge[source "{u}" target "{v}"]' for u, v in g.edges()
) + "]"

# `df` is the analytic cohort (one row per new initiator) with the named columns; latent nodes
# (true_severity, frailty) are intentionally absent because claims cannot measure them.
def select_adjustment_set(df: pd.DataFrame):
    model = CausalModel(
        data=df, treatment="treatment", outcome="AKI", graph=gml,
        common_causes=None,
    )
    estimand = model.identify_effect(proceed_when_unidentifiable=False)
    # Valid backdoor set = pre-index common causes only; descendants of treatment are excluded.
    backdoor = estimand.get_backdoor_variables()
    post_index = nx.descendants(g, "treatment")
    inadmissible = sorted(post_index)            # must NOT enter the PS
    latent_open = [n for n in ("true_severity", "frailty") if n not in backdoor]
    print("Adjustment set for the propensity score:", sorted(backdoor))
    print("Inadmissible (post-index descendants):", inadmissible)
    print("Unmeasured open backdoor paths -> need negative control / QBA:", latent_open)
    return sorted(backdoor)

library(dagitty)

# Latent nodes flagged [latent]; exposure/outcome flagged so adjustmentSets() can solve the backdoor.
g <- dagitty('dag {
  eGFR_proxy           -> treatment   eGFR_proxy           -> AKI
  diabetes_severity    -> treatment   diabetes_severity    -> AKI
  prior_utilization    -> treatment   prior_utilization    -> AKI
  prescriber_preference -> treatment
  true_severity [latent] -> diabetes_severity   true_severity -> AKI
  frailty       [latent] -> prior_utilization    frailty       -> AKI
  treatment -> volume_depletion        volume_depletion      -> AKI   /* mediator */
  treatment -> adherence               adherence             -> AKI   /* descendant */
  treatment -> subsequent_utilization  AKI -> subsequent_utilization  /* collider */
  treatment [exposure]                 AKI [outcome]
  treatment -> AKI
}')

# Minimal sufficient set to identify the TOTAL effect of treatment on AKI (backdoor criterion).
adjustmentSets(g, type = "minimal", effect = "total")
# Variables that would bias the estimate if conditioned on:
print(setdiff(descendants(g, "treatment"), "treatment"))  # post-index mediators/colliders

# Implied conditional independencies the DAG asserts, then falsify them against the cohort.
# `cohort` is the analytic data frame (one row per initiator) holding the OBSERVED node columns.
impliedConditionalIndependencies(g)
localTests(g, data = cohort, type = "cis.loess", R = 200)  # |estimate| far from 0 => DAG misspecified