QC, Double Programming, and Reproducible Analysis

In plain language

In real-world studies, the same data can produce different answers depending on how the analysis code handles ambiguous cases — like a patient who enrolled and left a health plan on the same day. Double programming is a check where a second analyst independently writes their own code to reproduce the same results, without looking at the first analyst's code; any difference in the numbers gets documented and traced back to its source. Reproducibility engineering is the set of practices — saving exactly which software versions were used, freezing the data with a checksum, and setting random seeds — that allow anyone to re-run the same analysis in the future and get the exact same answer.

The QC ladder in real-world evidence analysis

Every RWE analysis output — a cohort count, an incidence rate, a hazard ratio — is only as trustworthy as the code that produced it. Quality control (QC) in analytic programming is not a final box-check; it is a tiered system of verification activities that begins when the first line of code is written and ends when the locked output is delivered. The industry-standard ladder runs from light to rigorous.

Code review

is the first rung: a second analyst reads the programming code against the SAP to confirm that each step — cohort-definition filters, outcome algorithm, covariate coding, model specification — matches what was pre-specified. Code review catches obvious deviations and is inexpensive, but it is vulnerable to shared misreadings. If both the author and reviewer misinterpret the same SAP sentence in the same direction, the error survives intact.

Output review

is the second rung: a senior analyst or statistician reviews the formatted tables, listings, and figures against expected ranges drawn from the literature and clinical judgment. Are the incidence rates plausible? Is the hazard ratio directionally consistent with prior estimates? Output review catches gross numerical implausibilities but not subtle implementation errors that produce plausible-looking wrong numbers.

Independent double programming

is the highest rung and the only one that systematically catches errors arising from ambiguity between the SAP and the code. A second programmer — who has not seen the primary programmer's code — reads the SAP and independently writes code to re-derive the same key outputs (cohort counts, baseline characteristics, primary endpoint estimates). The two output sets are compared numerically, typically using a formal comparison utility (PROC COMPARE in SAS, a diff of formatted tables, or a purpose-built reconciliation script). Any discrepancy triggers a discrepancy resolution log: both programmers document their interpretation of the SAP clause that was ambiguous, the discrepancy is traced to its data-level source, and the correct interpretation is confirmed with the study statistician. If the SAP was ambiguous, it is amended before outcome-dependent re-programming begins.

When full double programming is warranted versus risk-based QC

Full independent double programming is the expected standard for regulatory deliverables — FDA submissions, EMA dossiers, PMDA safety reports — and for the primary efficacy and safety endpoints of any comparative study intended to influence formulary coverage, label language, or clinical guideline decisions. The cost in analyst time is justified because a programming error discovered after regulatory submission requires a complete re-analysis and formal amendment, costing far more than the original double-programming step.

For internal exploratory analyses, feasibility counts, and hypothesis-generating work where results will not directly influence a regulated decision, a risk-based QC approach is appropriate: code review plus output review, with a documented rationale for not double programming. The key discipline is that the decision is explicit and archived, not simply an omission. If an exploratory finding is later repurposed as the basis for a regulatory or HTA deliverable, the QC level must be escalated retrospectively — meaning the analysis must be re-done with independent double programming, not simply certified after the fact.

Reproducibility engineering for RWE

Independent double programming confirms that two programmers reach the same number today. Reproducibility engineering ensures the same number can be reached tomorrow, by any analyst, from the raw data. The minimum reproducibility stack for a RWE analysis has five components.

First, version control (git): every analysis script, macro, and configuration file is committed to a version-controlled repository with meaningful commit messages. Tags or branches mark the code state used to produce each versioned deliverable. Without version control, there is no reliable way to identify which code produced which output if a discrepancy surfaces months after delivery.

Second, locked data snapshots with checksums: the raw data extract is frozen at a defined calendar cut date and a cryptographic checksum (SHA-256 or MD5) is computed for every source file. The checksum is stored alongside the data manifest and re-verified at the start of every re-run. In living databases — administrative claims systems that are continuously updated with late-arriving adjudicated claims — the cut date is not administrative convenience. It is part of what the analysis is estimating. Two analyses that use identical code and identical study parameters but different cut dates may produce numerically different cohort sizes and outcome counts because late-arriving claims can change who is observable in the data. The cut date must therefore be documented with the same precision as the eligibility criteria, and some authors argue it should be considered part of the estimand specification for living databases.

Third, environment pinning: software package versions change analytical results. The `survival` package's default tie-handling method for the Cox model shifted between major R versions, silently changing baseline hazard estimates. SAS PROC PHREG option defaults differ across SAS release years. Python `lifelines` and `statsmodels` have introduced breaking changes to default arguments across versions. Pinning the software environment — via `renv.lock` in R, `requirements.txt` or `pyproject.toml` with pinned hashes in Python, or a documented SAS metadata server version in a regulated environment — ensures that a re-run in two years uses identical software. Containerization (Docker, Apptainer/Singularity) provides the strongest guarantee by freezing the operating system, language runtime, and all dependencies in a single portable image.

Fourth, seeds for stochastic operations: any analysis step that involves randomness — bootstrap confidence intervals, multiple imputation, propensity-score matching with random tiebreaking, Monte Carlo sensitivity analysis — must have an explicit random seed set immediately before execution. The seed value must be stored in the analysis script itself (not only in a log file), and documented in the SAP. Without a seed, two runs of the same code on the same data will produce different numbers, making numeric comparison between programmer A and programmer B impossible for stochastic steps.

Fifth, one-button rerun from raw extract to tables: the gold standard is a single shell command — a `Makefile`, a `targets` pipeline in R, a `_targets.R`, a numbered SAS flow script, or a Snakemake workflow — that executes the full analysis chain from reading the checksummed raw data through cohort construction, variable derivation, statistical modeling, and table/figure generation without any manual intervention or analyst-specific path edits. This makes the analysis independently auditable: a reviewer can clone the repository, restore the pinned environment, point to the locked data extract, and reproduce every table.

The analytic-decision audit trail

Every choice that changes who is in the cohort — washout length, same-day-event handling, grace period definition, lookback window, inclusion of patients with zero-day enrollment — must be logged in the analytic-decision audit trail. This log is distinct from the git commit history (which captures what changed in the code) and distinct from the discrepancy resolution log (which captures what changed in response to QC findings). The audit trail documents the reasoning for each design and implementation choice: what the SAP said, what edge cases were found in the data distribution, and what the study statistician decided. The audit trail ties directly to attrition reporting: a CONSORT-style flow diagram showing how many patients were excluded at each step must be numerically consistent with the audit trail, and every filter must appear in both.

Common silent failures QC catches

The most dangerous programming errors in RWE are those that produce plausible-looking numbers. Independent double programming and systematic output review consistently surface four archetypes.

Merge duplications (claim-line fan-out): a patient with five claim lines for the same service date merges five rows into a utilization table instead of one, inflating cost totals and event counts. The symptom is a cohort-level mean cost that is implausibly high relative to literature benchmarks, or a post-merge row count that exceeds the pre-merge patient count. The fix is a deduplication step before the join, or a many-to-one assertion after it.

Missing-data coercion: an unhandled NULL or SAS missing value (`.`) is coerced to zero in arithmetic, making a patient appear to have zero cost or zero utilization rather than missing data. The coercion is silent: the analysis runs without error, but missing costs are counted as zero costs in the mean. In R, `NA` propagation rules differ from SAS's `.` conventions; a port from SAS to R that does not explicitly handle missingness will silently shift means downward.

Date arithmetic off-by-ones: calculating enrollment duration as `disenrollment_date minus enrollment_start` gives the number of days between the two dates, not the number of days of enrollment (which includes the start date and thus adds 1 in the inclusive convention). The CONSORT flow count and the eligibility criterion must agree on which convention is used, and that convention must be written into the SAP.

Unintended row filtering: a `WHERE` clause that correctly filters the base population is applied after a merge that changed the grain of the dataset, inadvertently dropping rows that should be retained or retaining duplicates that should have been collapsed. The symptom is a row count that does not match the expected attrition step in the flow diagram.

ALCOA+ data integrity in plain terms

The FDA's data integrity guidance for regulated submissions uses the ALCOA+ framework: Attributable, Legible, Contemporaneous, Original, Accurate, plus Complete, Consistent, Enduring, and Available. For an RWE analysis, these translate to: every data transformation step is documented and traceable to the analyst who made it (Attributable); code is version-controlled and retrievable (Legible, Enduring); checksums verify that the raw data have not been altered since the cut date (Original, Accurate); the locked dataset policy ensures the analysis dataset matches what was used to produce the submitted output (Consistent, Complete); and the code repository and data manifest remain accessible for the regulatory retention period (Available).

QC for AI-assisted analysis code

When analysis code is generated or accelerated by a large language model, the same independent verification standard applies. LLM-generated code is not exempt from double programming because its origin differs from a human programmer's. In practice, LLM-generated code for RWE analysis should be treated as a first-draft submission from a junior programmer: it requires code review against the SAP, testing on synthetic or subsetted data with known outputs, and — for primary endpoints in regulatory deliverables — independent double programming. LLM-generated code has characteristic failure modes that parallel those of human code: silent off-by-one errors in date arithmetic, merge joins that fan out rows, and eligibility criteria implemented from a common reading of similar language rather than the specific SAP wording. These are precisely the errors that independent double programming is designed to detect.

Pros, cons, and trade-offs

Independent double programming: - Pros: highest sensitivity for detecting errors arising from SAP ambiguity; produces a discrepancy resolution log that is itself a regulatory audit artifact; identifies SAP gaps before outcome data are examined; the only QC method that independently tests the interpretation of eligibility criteria. - Cons: approximately doubles the programming effort; requires two analysts with comparable technical depth; cannot catch errors that both programmers make identically because they share the same misreading of the SAP.

Risk-based QC (code review plus output review): - Pros: substantially less expensive in analyst time; appropriate and sufficient for exploratory and hypothesis-generating work; can be applied consistently across high volumes of internal analyses. - Cons: misses errors arising from shared misinterpretation; output review may fail to detect plausible-but-wrong numbers in domains where benchmarks are wide.

Reproducibility stack (version control, snapshots, pinning, seeds): - Pros: enables independent reproduction by any analyst at any future time; checksum-verified data snapshots provide chain-of-custody evidence for regulatory retention; pinned environments prevent silent result drift from package updates; seeds make stochastic outputs deterministic. - Cons: initial setup overhead; container builds add infrastructure complexity; renv/pyproject lockfiles require discipline to keep current; teams unfamiliar with git require training.

When to use

Apply the full QC ladder — including independent double programming — for any RWE analysis where outputs will be submitted to a regulatory agency, included in an HTA dossier, used to support a formulary or coverage decision, or published as the primary effectiveness or safety finding of a comparative study. Apply the reproducibility stack universally: even exploratory analyses benefit from version control and locked data snapshots because a finding that starts as exploratory may later be elevated to a confirmatory deliverable. Apply explicit seeds to every analysis that includes any stochastic step, regardless of the QC tier.

When NOT to use

Do not skip QC under time pressure — an undetected programming error in a regulatory submission requires a complete re-analysis and formal amendment, costing far more time than the original double-programming step. Do not treat LLM-generated code as pre-verified or as exempt from the QC ladder. Do not conflate code review with independent double programming: reviewing the same programmer's code does not test whether the SAP was interpreted correctly, because the reviewer may share the programmer's interpretation. Do not omit seeds from stochastic steps on the assumption that differences will be small — for bootstrap or multiple imputation with small effective sample sizes, seed-dependent variation can be clinically meaningful.

Interpreting the output

The primary deliverable of a QC process is not a p-value or an effect estimate — it is a discrepancy resolution log paired with a reconciliation statement confirming that both programmers reach numerically identical outputs after all discrepancies are resolved. In the worked example, Programmer A reports a cohort of n = 12,450 and Programmer B reports n = 12,480.

(1) Formal interpretation. The discrepancy of 12,480 minus 12,450 equals 30 patients. These 30 patients have enrollment_start equal to disenrollment_date — they enrolled and disenrolled on the same calendar day, yielding an enrollment duration of zero days when computed as the arithmetic difference between the two dates. Programmer A applied the filter requiring strictly positive enrollment duration (disenrollment_date greater than enrollment_start), which excluded all 30 patients. Programmer B applied the filter requiring a non-null enrollment_start, which retained all 30 patients. Both implementations are internally consistent with a plausible reading of the SAP's phrase "continuously enrolled during the pre-index period." The discrepancy is a SAP ambiguity, not a programming bug in either implementation. Resolution requires a SAP amendment that specifies the minimum enrollment duration explicitly.

(2) Practical interpretation. For a study statistician reviewing this discrepancy report: the 30 affected patients represent a very small fraction of the total cohort. The correction reduces the cohort to 12,450 and does not change the direction of the analysis. However, the SAP must be amended before outcome data are examined, and the CONSORT-style attrition flow diagram must be updated to show these patients as excluded under the clarified criterion. Even a discrepancy of this magnitude is meaningful for a regulatory deliverable — it demonstrates that the SAP contained a boundary condition that was not operationally specified, and the discrepancy resolution log is the audit evidence that the ambiguity was identified and resolved prospectively.

Worked example

Scenario

Two programmers are independently implementing the eligibility algorithm for a retrospective claims cohort study of a new oral anticoagulant. The SAP requires patients to be "continuously enrolled in the 6-month pre-index period." After each programmer delivers their cohort-build code and a table of n, the QC reviewer compares the outputs and flags a discrepancy: Programmer A reports a cohort of 12,450 patients, Programmer B reports 12,480. The QC reviewer opens the discrepancy resolution log and asks both programmers to document their implementation of the continuous enrollment criterion.

Dataset

Four sample rows from the enrollment table representing the 30 discrepant patients — all have enrollment_start equal to disenrollment_date, yielding zero days when enrollment duration is computed as the arithmetic difference. Programmer A excludes these patients; Programmer B retains them.

Steps

• Programmer A implements the enrollment filter as: keep patients where disenrollment_date > enrollment_start (strictly greater than, so enrollment duration > 0 days). All 30 patients with same-day enrollment are excluded. Programmer A cohort = 12450.

• Programmer B implements the enrollment filter as: keep patients where enrollment_start IS NOT NULL (any non-missing enrollment record counts as enrolled). All 30 same-day patients pass this filter. Programmer B cohort = 12480.

• Discrepancy reported. Difference: 12480 - 12450 = 30 patients.

• Both programmers inspect the 30 patients present in B's cohort but absent from A's. Every one of the 30 has enrollment_start equal to disenrollment_date, so days_enrollment_diff = 0 for each.

• SAP clause reviewed: "continuously enrolled in the 6-month pre-index period" does not define the minimum enrollment duration or specify how to handle same-day enrollment records. The boundary condition of zero-day enrollment is a SAP ambiguity — both implementations are defensible readings of the existing language.

• The study statistician is consulted. Decision: same-day enrollment records represent administrative artifacts (enrollment initiated and immediately reversed). The SAP is amended to require enrollment_duration > 0 days (disenrollment_date strictly greater than enrollment_start). Both programmers re-run. Both reach 12450. Discrepancy log closed.

Result

Programmer A = 12450 patients; Programmer B = 12480 patients; discrepancy = 12480 - 12450 = 30. Root cause: 30 patients with same-day enrollment (days_enrollment_diff = 0) were handled differently under two valid interpretations of the SAP enrollment criterion. Resolution: SAP amended to require strictly positive enrollment duration; both programmers reach 12450 after re-run. Attrition flow diagram updated to show 30 patients excluded under the clarified criterion.

Runnable example

import hashlib
import json
import sys
from datetime import date
from pathlib import Path

import numpy as np
import pandas as pd

# ── 1. SHA-256 checksum a data extract file ────────────────────────────────────────
def sha256_file(path: str) -> str:
    """Return the SHA-256 hex digest of a file. Re-verify before every analysis run."""
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(65_536), b""):
            h.update(chunk)
    return h.hexdigest()

# ── 2. Build and save a locked data snapshot manifest ─────────────────────────────
def create_snapshot_manifest(data_files: list[str],
                             manifest_path: str = "snapshot_manifest.json") -> dict:
    """
    Log the cut date and SHA-256 hash of every source file.
    Store manifest in the git repo alongside the analysis code.
    """
    manifest = {
        "cut_date": str(date.today()),
        "python_version": sys.version,
        "files": {p: sha256_file(p) for p in data_files},
    }
    with open(manifest_path, "w") as f:
        json.dump(manifest, f, indent=2)
    print(f"Snapshot manifest created: {len(data_files)} files, cut_date={manifest['cut_date']}")
    return manifest

# ── 3. Verify snapshot integrity before analysis begins ────────────────────────────
def verify_snapshot(manifest_path: str = "snapshot_manifest.json") -> None:
    """
    Re-compute SHA-256 for every file in the manifest and compare.
    Raises RuntimeError if any file has changed since the snapshot was taken.
    """
    manifest = json.loads(Path(manifest_path).read_text())
    errors = []
    for path, expected in manifest["files"].items():
        actual = sha256_file(path)
        if actual != expected:
            errors.append(f"CHECKSUM MISMATCH: {path}")
    if errors:
        raise RuntimeError("Data integrity check FAILED:\n" + "\n".join(errors))
    print(f"Snapshot verified: {len(manifest['files'])} files OK (cut_date={manifest['cut_date']})")

# ── 4. Merge row-count validation (catch claim-line fan-out) ───────────────────────
def safe_merge(left: pd.DataFrame, right: pd.DataFrame,
               on: str | list, how: str = "left", label: str = "merge") -> pd.DataFrame:
    """
    Merge left and right, then assert that no patient was duplicated.
    A post-merge row count exceeding the pre-merge patient count signals fan-out from
    duplicate keys in the right table (common with claim-line-level cost tables).
    """
    n_before = len(left)
    result = left.merge(right, on=on, how=how)
    n_after = len(result)
    if n_after > n_before:
        raise ValueError(
            f"QC FAIL [{label}]: merge inflated rows {n_before} -> {n_after} "
            f"(+{n_after - n_before}). Deduplicate right table before joining."
        )
    print(f"QC OK [{label}]: {n_before} -> {n_after} rows "
          f"({n_before - n_after} unmatched records dropped in left-join)")
    return result

# ── 5. Set seeds before any stochastic step ────────────────────────────────────────
ANALYSIS_SEED = 20240301  # document this value in the SAP; do not change post-lock

def set_global_seeds(seed: int = ANALYSIS_SEED) -> None:
    """Set Python random and NumPy seeds. Call once at the start of the analysis script."""
    import random
    random.seed(seed)
    np.random.seed(seed)
    # For scikit-learn: pass random_state=seed to every estimator
    # For scipy bootstrap: rng=np.random.default_rng(seed)
    print(f"Seeds set to {seed} (ANALYSIS_SEED = {seed})")

# ── 6. Enrollment duration edge case: same-day enrollment detection ────────────────
def flag_zero_day_enrollments(enroll_df: pd.DataFrame,
                              start_col: str = "enrollment_start",
                              end_col: str = "disenrollment_date") -> pd.DataFrame:
    """
    Compute enrollment duration and flag same-day patients (duration = 0).
    SAP must specify whether to keep (Programmer B convention) or exclude (Programmer A).
    """
    df = enroll_df.copy()
    df[start_col] = pd.to_datetime(df[start_col])
    df[end_col]   = pd.to_datetime(df[end_col])
    df["days_enrolled"] = (df[end_col] - df[start_col]).dt.days  # arithmetic difference
    df["same_day_flag"] = df["days_enrolled"] == 0
    n_same_day = df["same_day_flag"].sum()
    print(f"Zero-day enrollment records: {n_same_day} "
          f"({'excluded' if True else 'retained'} per strict-positive-duration rule)")
    return df

# ── 7. Discrepancy report: compare two cohort count vectors ────────────────────────
def compare_outputs(prog_a: dict, prog_b: dict, tolerance: float = 0.0001) -> list[str]:
    """
    Compare Programmer A and Programmer B output dicts.
    Returns a list of discrepancy strings; empty list = no discrepancy.
    """
    discrepancies = []
    all_keys = set(prog_a) | set(prog_b)
    for key in sorted(all_keys):
        a_val = prog_a.get(key)
        b_val = prog_b.get(key)
        if a_val is None or b_val is None:
            discrepancies.append(f"{key}: missing in one output (A={a_val}, B={b_val})")
        elif isinstance(a_val, (int, float)) and abs(a_val - b_val) > tolerance:
            discrepancies.append(f"{key}: A={a_val}, B={b_val}, diff={b_val - a_val}")
    return discrepancies

# ── Demo: reproduce the worked-example discrepancy ────────────────────────────────
if __name__ == "__main__":
    prog_a_output = {"cohort_n": 12450, "mean_age": 63.2, "pct_female": 0.487}
    prog_b_output = {"cohort_n": 12480, "mean_age": 63.1, "pct_female": 0.486}
    issues = compare_outputs(prog_a_output, prog_b_output, tolerance=0.005)
    if issues:
        print("DISCREPANCIES FOUND — open discrepancy resolution log:")
        for d in issues:
            print(f"  {d}")
    # cohort_n: A=12450, B=12480, diff=30
    # 12480 - 12450 = 30  -> same-day enrollment boundary case

# ── 1. Environment locking with renv ──────────────────────────────────────────────
# At project setup:  renv::init(); renv::snapshot()   -> writes renv.lock
# At re-run:         renv::restore()                  -> installs exact versions
# renv.lock pins R version, package versions, and CRAN snapshot date.
# Commit renv.lock to git so every analyst uses identical software.

# ── 2. Assert survival package version (default tie method changed across versions) ─
stopifnot(
  "survival >= 3.5 required; earlier versions use different default tie method for Cox" =
    packageVersion("survival") >= "3.5"
)

# ── 3. Set seeds before any stochastic step ───────────────────────────────────────
ANALYSIS_SEED <- 20240301L   # document in SAP; do not change after analysis lock
set.seed(ANALYSIS_SEED)
# For mice (multiple imputation):
#   imp <- mice::mice(data, m = 20, seed = ANALYSIS_SEED)
# For boot (bootstrap):
#   set.seed(ANALYSIS_SEED); boot_res <- boot::boot(data, stat_fn, R = 1000)
# For MatchIt (propensity matching with random tiebreaking):
#   set.seed(ANALYSIS_SEED); m_out <- matchit(formula, data, method = "nearest")

# ── 4. Merge row-count validation (catch claim-line fan-out) ─────────────────────
safe_merge <- function(left, right, by, all.x = TRUE, label = "merge") {
  n_before <- nrow(left)
  result   <- merge(left, right, by = by, all.x = all.x)
  n_after  <- nrow(result)
  if (n_after > n_before) {
    stop(sprintf(
      "QC FAIL [%s]: merge inflated rows %d -> %d (+%d). Deduplicate right table.",
      label, n_before, n_after, n_after - n_before
    ))
  }
  message(sprintf("QC OK [%s]: %d -> %d rows", label, n_before, n_after))
  result
}

# ── 5. Enrollment duration: flag zero-day records ─────────────────────────────────
flag_zero_day_enrollments <- function(df,
                                      start_col = "enrollment_start",
                                      end_col   = "disenrollment_date") {
  df[[start_col]] <- as.Date(df[[start_col]])
  df[[end_col]]   <- as.Date(df[[end_col]])
  # Arithmetic difference (days between dates, NOT inclusive of start day)
  df$days_enrolled <- as.integer(df[[end_col]] - df[[start_col]])
  df$same_day_flag <- df$days_enrolled == 0L
  n_same <- sum(df$same_day_flag, na.rm = TRUE)
  message(sprintf("Zero-day enrollment records: %d (SAP must specify: include or exclude)", n_same))
  df
}

# ── 6. Discrepancy comparison: reproduce worked-example ───────────────────────────
prog_a <- list(cohort_n = 12450L, mean_age = 63.2, pct_female = 0.487)
prog_b <- list(cohort_n = 12480L, mean_age = 63.1, pct_female = 0.486)

compare_outputs <- function(a, b, tolerance = 0.005) {
  keys <- union(names(a), names(b))
  diffs <- character(0)
  for (k in keys) {
    av <- a[[k]]; bv <- b[[k]]
    if (is.null(av) || is.null(bv)) {
      diffs <- c(diffs, sprintf("%s: missing in one output (A=%s, B=%s)", k, av, bv))
    } else if (abs(av - bv) > tolerance) {
      diffs <- c(diffs, sprintf("%s: A=%s, B=%s, diff=%+.4f", k, av, bv, bv - av))
    }
  }
  diffs
}

discrepancies <- compare_outputs(prog_a, prog_b)
if (length(discrepancies) > 0) {
  message("DISCREPANCIES FOUND -- open discrepancy resolution log:")
  for (d in discrepancies) message("  ", d)
}
# cohort_n: A=12450, B=12480, diff=+30
# 12480 - 12450 = 30 -> same-day enrollment boundary case (see worked example)

# ── 7. One-command pipeline note ─────────────────────────────────────────────────
# Use {targets} for a reproducible pipeline:
#   library(targets)
#   tar_make()   # re-runs only changed steps; skips up-to-date targets
# targets integrates with renv and stores output hashes to detect stale outputs.