LLM-Assisted Data Abstraction and Evidence Work in RWE

In plain language

Large language models (LLMs) like GPT-4 can read clinical notes and pull out structured information — for example, whether a patient's tumor has a certain mutation — far faster than a human abstractor can do the same work for thousands of patients. But LLMs make mistakes in characteristic ways: they sometimes invent plausible-sounding values that do not appear in the note, miss negations ("EGFR negative" read as positive), or confuse dates. Because of these errors, every LLM extraction pipeline must be checked against a sample of manually reviewed records to measure how often it is right — the same positive predictive value and sensitivity calculation used to validate any other claims or EHR algorithm — and any fields the LLM is uncertain about must go to a human reviewer before they enter an analysis.

Where LLMs are entering RWE practice

Large language models are being deployed across several distinct evidence-generation tasks. Chart abstraction at scale: extracting structured fields — EGFR mutation status, cancer staging, biomarker values, disease progression dates — from unstructured clinical notes, discharge summaries, and pathology reports, using structured-output prompting that forces the model to return a JSON schema rather than free prose. Systematic literature review (SLR) acceleration: first-pass title/abstract screening against inclusion and exclusion criteria, and structured data extraction from full-text articles (effect sizes, sample sizes, covariate lists). Protocol and SAP drafting: generating first-draft sections from a study concept or a prior RWE protocol template, which a human reviewer then edits and approves. Analysis code generation: scaffolding SAS, R, or Python code for standard RWE steps (cohort construction, PS estimation, outcome ascertainment) from a structured SAP description. Terminology mapping: suggesting ICD-10, NDC, or LOINC codes for a free-text concept, which a clinical informaticist then accepts or overrides. In every one of these roles, the LLM functions as a powerful draft generator — not as a final decision-maker.

The governing principle: no load-bearing LLM output

The foundational rule in any LLM-augmented RWE workflow is that an LLM's output is never the end of the pipeline. Every structured field extracted from a note, every inclusion decision in an SLR screen, and every code suggestion must pass through at least one deterministic check or human review gate before it enters an analysis. This is not a counsel of excessive caution: it reflects the measurement-error framing that RWE already applies to all algorithm-defined variables. An LLM extractor is a phenotyping algorithm. It has operating characteristics — sensitivity, specificity, PPV — that must be measured against a chart-review gold standard, reported, and used to assess or correct for misclassification, exactly as described in the algorithm-validation and endpoint-adjudication-chart-review literatures. A model's stated accuracy on a benchmark is not its operating characteristic on your notes, your note templates, or your study population.

Hallucination modes specific to clinical abstraction

LLMs fail in characteristic ways that differ from those of rule-based algorithms. The most consequential failure modes in RWE abstraction are: (1) Plausible-but-absent values — the model returns a biologically plausible EGFR status or a credible-looking lab value that does not appear anywhere in the source document; the value is coherent enough to evade casual review. (2) Unit errors — a creatinine value is returned in mg/dL when the note reports mmol/L, or a dose is extracted in mg when the chart uses mcg; these are particularly dangerous when downstream code performs threshold comparisons. (3) Date confabulation — progression dates, biopsy dates, or treatment-start dates that are shifted by weeks or months relative to the documented event; the model anchors to the most recent date it encountered in context rather than the semantically correct one. (4) Negation misses — returning a value as present when the note states "EGFR mutation negative" or "no evidence of progression"; negation handling remains a systematic weakness even in state-of-the-art models when the negation is embedded in a complex clinical sentence. These failure modes persist even in strong models, are partially mitigated by chain-of-thought prompting or span citation requirements, and are best controlled by systematic validation rather than model selection alone.

Design patterns that work

Several engineering patterns reduce LLM error in RWE abstraction. Structured output schemas: constrain the model to return only the fields defined in a JSON schema with allowed values (e.g., EGFR_status: "positive" | "negative" | "indeterminate" | "not_ reported"); open-ended text generation is the highest-error configuration. Span-citation requirements: require the model to quote the exact source text that supports each extracted field (a "citation span"); a field with no supporting quote is flagged for human review rather than accepted. Dual-model or model-versus-rule adjudication: run two independent LLM passes (or an LLM pass plus a rule-based extractor) and send disagreements to a human adjudicator; this mirrors the two-reviewer-plus-tiebreaker structure in chart adjudication. Confidence-thresholded human review queues: use the model's log-probability or a downstream classifier's confidence score to route low-confidence extractions to a human reviewer and high-confidence extractions to automated acceptance, then validate that the automated tier meets a pre-specified PPV floor. Batch-versus-interactive cost design: processing 10,000 notes at batch API rates is orders of magnitude cheaper than interactive queries; cost modeling should be done before committing to a dataset scope.

Evaluation discipline: the same rigor as any other algorithm

The evaluation framework for an LLM abstraction pipeline mirrors the algorithm-validation framework exactly. A held-out gold standard is required: a randomly sampled set of source documents that have been independently annotated by a human expert (clinician or trained abstractor) and set aside before any prompting. Against this gold standard, compute PPV (what fraction of the model's positive calls are correct), sensitivity (what fraction of true positives the model found), and Cohen's kappa (inter-rater agreement between model and gold annotator). Stratify performance by note type (discharge summary vs. progress note vs. pathology report), by note length, and by writing style — models degrade systematically on non-standard templates or high-complexity notes. Drift monitoring is mandatory: a model update, an API version change, or a shift in the clinical note templates used by the study sites is a data-pipeline change that requires revalidation on the existing gold standard. Version-pin the model at the time of study conduct and document the exact model identifier in the SAP; failing to pin means the extraction behavior is unknowable at the time of regulatory review.

Regulatory posture

FDA's evolving guidance on AI and machine learning in drug development (the Action Plan for AI/ML-based software as a medical device, the 2023 discussion paper on AI in drug development) consistently emphasizes three requirements that apply directly to LLM use in RWE: (1) transparency — the model, version, prompt template, and extraction schema must be documented in the SAP appendix; (2) audit trails — every LLM decision must be logged with its inputs, outputs, and any human review outcome so the analysis can be reproduced or audited; (3) human-in-the-loop documentation — the protocol must specify which decisions are automated and which require human review, what the human review criteria are, and who conducted the review. For HTA submissions, the same standard applies under NICE's real-world evidence framework and EMA's guidance on complex clinical trials data. The practical implication: an LLM used only for draft generation with full human review of every output sits at the lowest regulatory risk level; an LLM used for automated data abstraction that feeds directly into an analysis sits at the highest and requires a full validation study.

Privacy and PHI handling

Clinical notes contain protected health information (PHI) under HIPAA and equivalent regulations. Before sending notes to any cloud API (OpenAI, Anthropic, Google, etc.), a Business Associate Agreement (BAA) must be in place with the vendor, and the specific API product must be BAA-eligible (consumer-tier products are typically not). For studies where a BAA cannot be obtained — or where the risk profile or data governance policy prohibits any external transmission — locally deployed open-weight models (Llama 3, Mistral, clinical fine-tunes such as BioMistral) offer an alternative that keeps PHI on-premises, at the cost of lower out-of-the-box performance and the operational overhead of model hosting and version management. De-identification of notes before external API use is a mitigation option but introduces its own error rate and requires validation.

Pros, cons, and trade-offs

Pros: LLMs can scale structured data extraction from unstructured notes to cohorts of thousands or tens of thousands of patients — a task that is prohibitively expensive using only human abstractors. For SLR screening, LLMs can reduce the human screening burden by 60-80% on first-pass abstract review while maintaining recall close to 100% when a conservative confidence threshold is applied. Draft generation for protocols, SAPs, and report sections accelerates cycle times and reduces blank-page friction for study teams. Code generation for standard RWE pipelines reduces implementation error on boilerplate tasks and is immediately testable against expected outputs.

Cons: LLM performance on a general benchmark does not predict performance on a specific study's notes; validation against a study-specific gold standard is required and has real cost. Hallucination modes are non-random — they are systematic and correlated with note characteristics — so a model that looks accurate on aggregate metrics can fail badly on a specific note subtype. Regulatory audit trails add infrastructure overhead. PHI transmission to cloud APIs creates compliance risk and requires legal review. Model updates break reproducibility unless the model is pinned and the abstraction is re-run on updates.

Trade-offs: The core trade-off is scale vs. certainty. Human-only abstraction is accurate but limits cohort size; LLM-assisted abstraction scales but introduces quantifiable and manageable error. The correct analytic response to that error is not to hide it but to measure it (PPV/sensitivity validation), propagate it (quantitative bias analysis), and mitigate it (human review queues for low-confidence extractions). The concepts of endpoint adjudication and algorithm validation — not LLM engineering — are the methodological home for managing LLM abstraction quality in RWE.

When to use

Use LLM-assisted abstraction when the target structured fields are clearly defined and can be expressed as a constrained output schema; when the study cohort is too large for full manual abstraction; when a validation substudy against a chart-review gold standard is feasible and its cost is budgeted; when the note corpus is homogeneous enough that a single prompt template covers the majority of cases; and when a human review queue for low-confidence or high-stakes extractions is operationally possible. For SLR screening, use LLMs for first-pass abstract triage when the recall requirement is not absolute (or when a dual-reviewer structure provides the safety net). For code generation, use LLMs for standard pipeline scaffolding where the output will be reviewed and tested against expected results.

When NOT to use — and when LLM use is actively misleading

Do not treat LLM-extracted variables as validated without a gold-standard comparison study: a model that achieves 90% accuracy on a general benchmark can have a PPV of 0.60 on a specific note type in a specific study population, introducing outcome misclassification that biases effect estimates in an unknown direction. Do not use an LLM to extract the primary outcome of a regulatory study without a pre-specified validation plan, human review gate for every extracted value, and SAP-documented audit trail: the regulatory position on automated AI pipelines for primary efficacy/safety endpoints requires human-in-the-loop evidence. Do not use cloud APIs for PHI-containing notes without a valid BAA: even if the risk of a data breach is low, the compliance failure is not conditional on harm. Do not re-use a model prompt or version across time without checking whether the model or note templates have changed: a model update is a covariate shift that can silently change extraction behavior. Do not apply LLM code-generation output without testing it against known inputs and outputs: generated code commonly contains correct-looking but wrong logic for edge cases (date arithmetic, missing value handling, left-join vs inner-join semantics) that will not be caught by syntax checks alone.

Interpreting the output

In the worked example below, an LLM pipeline abstracts EGFR mutation status from 200 non-small cell lung cancer patient notes and is evaluated against a human-annotated gold standard. The 2x2 table yields: TP = 34, FP = 10, FN = 6, TN = 150. Accuracy = (34 + 150) / 200 = 184 / 200 = 0.92. Sensitivity = 34 / 40 = 0.85. PPV = 34 / 44 = 17 / 22 (approximately 0.773).

(1) Formal interpretation. The LLM extractor is an algorithm defined by its prompt template and model version. Its operating characteristics against the adjudicated gold standard are: sensitivity 0.85 (the model identified 34 of 40 true EGFR-positive patients; 6 true positives were missed, typically patients whose mutation was mentioned only in a pathology appendix or negated in complex syntax); PPV 34/44 = 17/22 (approximately 0.773 of the 44 patients the model called EGFR-positive are truly positive; 10 of 44 were false positives driven primarily by negation misses and plausible-but-absent values). These are the algorithm's operating characteristics in this note corpus at this model version; they are not portable to a different model version, a different institution's note templates, or a different primary cancer indication. Nondifferential misclassification with PPV below 1.0 will attenuate a risk ratio estimate toward the null; if the notes of patients on one treatment arm are systematically longer or more structured, the LLM may perform better in that arm, creating differential misclassification with unpredictable bias direction.

(2) Practical interpretation. For a study using EGFR status as a subgroup-defining covariate, a PPV of 0.773 means roughly 1 in 4 patients labeled EGFR-positive by the LLM pipeline is actually EGFR-negative — a level of subgroup contamination that could meaningfully attenuate the subgroup treatment effect estimate. Before proceeding to the comparative analysis, apply a sensitivity analysis that re-runs the primary model restricting to the high-confidence extraction tier (citations present, model confidence above threshold), and run a quantitative bias analysis propagating the validation PPV and sensitivity into the estimand. Document the validation design in the SAP, report the 2x2 performance table and its binomial confidence intervals, and specify the model identifier and prompt version in the methods section.

Worked example

Scenario

A real-world evidence team is studying treatment outcomes in non-small cell lung cancer (NSCLC). They need EGFR mutation status — positive, negative, or indeterminate — for a cohort of 200 patients. Because pathology reports are embedded in free-text notes, a clinical abstractor would normally read each note manually. Instead, the team uses a structured-output LLM prompt to extract EGFR status from each note, then validates the extraction against a human-annotated gold standard for all 200 patients. They compute accuracy, sensitivity, and positive predictive value (PPV) to decide whether the pipeline is acceptable for the subgroup analysis.

Dataset

Validation results for 200 NSCLC patient notes. Gold standard = human expert annotation of EGFR status; LLM = structured-output extraction result. A sample of the 2x2 contingency structure is shown; the full table underlies the computed metrics.

Steps

• Compile the full 200-patient 2x2 table from the validation run. Gold standard: 40 patients are truly EGFR-positive (gold-positive), 160 are truly EGFR-negative (gold-negative).

• LLM classifications: the model called 44 patients EGFR-positive. Of those 44, 34 were confirmed correct by the gold standard (true positives, TP = 34) and 10 were wrong (false positives, FP = 10). Of the 40 truly positive patients, the LLM missed 6 (false negatives, FN = 6). True negatives TN = 160 - 10 = 150.

• Compute accuracy (fraction of all 200 patients classified correctly): accuracy = (34 + 150) / 200 = 184 / 200 = 0.92.

• Compute sensitivity (fraction of truly EGFR-positive patients the LLM found): sensitivity = 34 / 40 = 0.85.

• Compute PPV (fraction of the LLM's positive calls that are actually positive): PPV = 34 / 44 = 17 / 22 (approximately 0.773). The 10 false positives were almost all negation misses — notes stating "EGFR mutation not detected" where the model extracted "EGFR-positive" because it anchored to the word "mutation" before reading "not detected."

• The 6 false negatives (sensitivity gap) came from patients whose mutation was documented only in a scanned pathology PDF referenced in the note but not transcribed as text — the LLM had no text to read, not a comprehension failure. This is an information-availability failure, not a model failure, and signals that the abstraction scope (text-only) needs to be documented clearly.

• Decide whether these operating characteristics are acceptable: sensitivity = 0.85 and PPV = 17/22 may be adequate for an exploratory subgroup analysis with a stated bias-analysis caveat, but would require a human review gate for the 44 LLM-positive calls before use in a regulatory or HTA submission.

Result

2x2 summary: TP = 34, FP = 10, FN = 6, TN = 150. Total correct = 34 + 150 = 184. accuracy = 184 / 200 = 0.92. sensitivity = 34 / 40 = 0.85. PPV = 34 / 44 = 17 / 22 (approximately 0.773). The LLM pipeline achieved high accuracy overall, but 1 in 4 of its EGFR-positive calls was a false positive — driven primarily by negation misses. A human review gate on all 44 LLM-positive calls, or a dual-model adjudication step, would recover a near-perfect PPV at modest additional cost before this variable enters a comparative analysis.

Runnable example

import json
import re
import numpy as np
import pandas as pd
from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from environment; substitute a BAA-eligible endpoint for PHI

# ── Structured-output schema for EGFR abstraction ──
SYSTEM_PROMPT = """
You are a clinical abstractor. Given a clinical note, extract the patient's EGFR mutation
status and return ONLY a JSON object matching this schema:
{
  "egfr_status": "<positive|negative|indeterminate|not_reported>",
  "citation_span": "<verbatim text from the note that supports the classification, or null>"
}
If no mention of EGFR is found, return "not_reported" with citation_span null.
If the note contains a negation (e.g. 'EGFR negative', 'no EGFR mutation detected'),
return "negative" and quote the negation phrase in citation_span.
""".strip()

def abstract_egfr(note_text: str, model: str = "gpt-4o-2024-08-06") -> dict:
    """Extract EGFR status from a single clinical note using structured-output prompting."""
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": note_text},
        ],
        response_format={"type": "json_object"},
        temperature=0,          # deterministic; required for reproducibility
        seed=42,                # pin for audit trail (OpenAI-specific)
    )
    raw = response.choices[0].message.content
    try:
        result = json.loads(raw)
    except json.JSONDecodeError:
        result = {"egfr_status": "parse_error", "citation_span": None}

    # Flag for human review: no citation span means the extraction is ungrounded.
    result["needs_human_review"] = (result.get("citation_span") is None
                                    and result.get("egfr_status") != "not_reported")
    result["model"] = model     # version pin for audit trail
    return result

def batch_abstract(notes: pd.DataFrame,
                   note_col: str = "note_text",
                   id_col: str = "patient_id",
                   model: str = "gpt-4o-2024-08-06") -> pd.DataFrame:
    """Run abstraction over a DataFrame of notes; returns one row per patient."""
    results = []
    for _, row in notes.iterrows():
        out = abstract_egfr(row[note_col], model=model)
        out[id_col] = row[id_col]
        results.append(out)
    return pd.DataFrame(results)

# ── Gold-standard validation ──
def validate_abstraction(results: pd.DataFrame,
                          gold: pd.DataFrame,
                          id_col: str = "patient_id",
                          gold_positive_label: str = "positive") -> dict:
    """
    Compute accuracy, sensitivity, and PPV against a gold-standard annotation.
    gold: DataFrame with columns [id_col, 'gold_egfr_status'] where gold_egfr_status
          is the adjudicated true value.
    """
    merged = results.merge(gold, on=id_col, how="inner")
    llm_pos = merged["egfr_status"] == gold_positive_label
    gold_pos = merged["gold_egfr_status"] == gold_positive_label

    tp = int((llm_pos & gold_pos).sum())
    fp = int((llm_pos & ~gold_pos).sum())
    fn = int((~llm_pos & gold_pos).sum())
    tn = int((~llm_pos & ~gold_pos).sum())
    n = tp + fp + fn + tn

    accuracy    = (tp + tn) / n  if n > 0 else np.nan
    sensitivity = tp / (tp + fn) if (tp + fn) > 0 else np.nan
    ppv         = tp / (tp + fp) if (tp + fp) > 0 else np.nan

    return {
        "n": n, "tp": tp, "fp": fp, "fn": fn, "tn": tn,
        "accuracy": round(accuracy, 4),
        "sensitivity": round(sensitivity, 4),
        "ppv": round(ppv, 4),
        # Flag count: how many extractions lack citation spans and need human review.
        "n_needs_human_review": int(results["needs_human_review"].sum()),
    }

# ── Example: reproduce the worked-example numbers ──
# With TP=34, FP=10, FN=6, TN=150 (n=200):
# accuracy = (34+150)/200 = 0.92, sensitivity = 34/40 = 0.85, PPV = 34/44 ~= 0.773

# ── Validate LLM abstraction against gold standard ──
# Input: results.csv with columns: patient_id, egfr_status, citation_span, needs_human_review
#        gold.csv   with columns: patient_id, gold_egfr_status

validate_abstraction <- function(results, gold,
                                 positive_label = "positive") {
  merged <- merge(results, gold, by = "patient_id", all = FALSE)
  llm_pos  <- merged$egfr_status       == positive_label
  gold_pos <- merged$gold_egfr_status  == positive_label

  tp <- sum( llm_pos &  gold_pos)
  fp <- sum( llm_pos & !gold_pos)
  fn <- sum(!llm_pos &  gold_pos)
  tn <- sum(!llm_pos & !gold_pos)
  n  <- tp + fp + fn + tn

  cat("2x2 table (LLM rows x Gold columns):\n")
  tab <- matrix(c(tp, fp, fn, tn), nrow = 2, byrow = FALSE,
                dimnames = list(c("LLM pos","LLM neg"), c("Gold pos","Gold neg")))
  print(tab)
  cat("\n")

  # Exact (Clopper-Pearson) confidence intervals via binom.test.
  ci <- function(num, den, label) {
    if (den == 0) { cat(label, "= NA (zero denominator)\n"); return(invisible(NULL)) }
    bt <- binom.test(num, den)
    cat(sprintf("%s = %d/%d = %.4f  95%% CI [%.4f, %.4f]\n",
                label, num, den, bt$estimate, bt$conf.int[1], bt$conf.int[2]))
  }

  ci(tp + tn, n,     "Accuracy   ")
  ci(tp,      tp+fn, "Sensitivity")
  ci(tp,      tp+fp, "PPV        ")

  cat(sprintf("\nExtractions lacking citation span (needs_human_review = TRUE): %d of %d\n",
              sum(results$needs_human_review, na.rm = TRUE), nrow(results)))

  invisible(list(tp=tp, fp=fp, fn=fn, tn=tn,
                 accuracy    = (tp+tn)/n,
                 sensitivity = tp/(tp+fn),
                 ppv         = tp/(tp+fp)))
}

# ── Reproduce worked-example numbers ──
# TP=34 FP=10 FN=6 TN=150 -> accuracy=(34+150)/200=0.92, Se=34/40=0.85, PPV=34/44
dummy_results <- data.frame(
  patient_id         = 1:200,
  egfr_status        = c(rep("positive", 44), rep("negative", 156)),
  needs_human_review = c(rep(FALSE, 34), rep(TRUE, 10), rep(FALSE, 156))
)
dummy_gold <- data.frame(
  patient_id       = 1:200,
  gold_egfr_status = c(rep("positive", 40), rep("negative", 160))
)
validate_abstraction(dummy_results, dummy_gold)
# Expected: accuracy = 184/200 = 0.92, sensitivity = 34/40 = 0.85, PPV = 34/44