Claims Payment Anatomy: Billed, Allowed, and Paid Amounts
Claims and encounter records can carry several distinct dollar fields for the same service — the billed (submitted) charge the provider asks for, the allowed amount the payer's fee schedule or contract recognizes, the plan-paid amount that is the payer's own liability, any secondary payer's own paid amount, and the patient's out-of-pocket responsibility (deductible, copay, coinsurance) — but not every field is populated on every record, and treating billed charges as a cost measure is the single most common analytic error in claims-based real-world cost research; which field is populated, and whether paid can legitimately be $0, further depends on whether the claim is fee-for-service, capitated Medicare Advantage/Medicaid-MCO encounter data, or a multi-payer extract requiring standardized repricing. A secondary payer's coordination-of-benefits (COB) payment is a separate payer liability, not part of the patient's own out-of-pocket responsibility, and must never be summed into it.
On this page
A single medical bill actually has several different dollar amounts attached to it, and they can differ by a factor of ten. There is the sticker price the hospital or doctor asks for (the billed charge), the lower price the insurance company's contract actually recognizes (the allowed amount), the part the insurance company itself pays (the plan-paid amount), and the part the patient owes (deductible, copay, and coinsurance). The single most common beginner mistake in claims-data cost research is grabbing the billed charge and calling it "the cost" — it is closer to a list price than a real transaction, and using it can overstate true cost several-fold.
Several numbers, one service line — and they are not interchangeable
An adjudicated claim line can carry a small family of dollar fields that answer different questions: what did the provider ask for, what did the payer's contract or fee schedule recognize, what did the payer itself pay, what did any secondary payer pay, and what does the patient owe. Not every field is populated on every record — availability depends on the data source, payer type, and adjudication status (see below). Analysts trained on clinical data instinctively reach for "the cost" as if it were a single number; claims data rarely hands you one without this context. The correct field depends entirely on the perspective the research question requires — system cost, payer cost, or patient burden — and using the wrong one, or conflating a transaction-price/expenditure measure with the provider's true economic cost, can distort resource estimates several-fold.
Billed (submitted) charges — the chargemaster problem
The billed charge (also called the submitted charge or, for institutional claims, the amount in UB-04 FL47) is the price the provider's chargemaster — an internal, largely arbitrary list-price file — assigns to the service before any contract or fee schedule is applied. Chargemaster prices are set unilaterally by the facility, vary enormously by facility market power, service line, and calendar year, and bear only a loose relationship to the cost of the resources actually consumed. Bai and Anderson's audit of hospital charge-to-cost ratios found an average (mean) charge-to-cost ratio of 3.4 and a mode of 2.4 across the hospitals studied, with the 50 highest-markup hospitals in the sample billing close to 10 times their Medicare-cost-report-based costs, and Cooper et al.'s claims-based analysis of privately insured prices found comparable multi-fold variation in what was actually allowed across hospitals for the same service — variation that billed charges alone cannot even begin to explain, because almost no payer pays anywhere close to the charge. Billed charges are a list price, not a negotiated transaction price, and are a poor default proxy for resource cost or system spend: do not use them as a cost, budget-impact, or cost-effectiveness measure without a validated conversion (e.g., an appropriate cost-to-charge ratio) or a research question specifically about charge-based exposure (uninsured or out-of-network patient liability, balance billing, charge-based contracts). They remain the uninformative-but-present starting point before adjudication, and for out-of-network or uninsured patients they can determine actual liability directly rather than merely serving as a pricing artifact.
Allowed amount — the plan-negotiated price, and the correct default for "system cost"
The allowed amount (also called the negotiated rate, the contracted rate, or the plan allowance) is what the payer's fee schedule or negotiated contract recognizes as the total transaction price for the service, before splitting that liability among the primary payer, any secondary payer, and the patient. For commercial claims this is the output of the plan-provider contract; for Medicare fee-for-service (FFS) it is the output of a standardized payment system (MS-DRG, APC, RBRVS fee schedule) plus geographic and other adjustments. For a final-action, in-network claim, the allowed amount is generally the standard proxy for "cost to the system" in comparative cost studies, because it approximates the actual transaction price the provider agreed to accept — but it is a transaction-price/expenditure measure, not the provider's true economic cost, and it does not always bound what the patient can ultimately be billed: out-of-network care can be balance-billed above the allowed amount, and noncovered services are not represented in it at all. On a final-action, in-network FFS line where the data dictionary defines "allowed" as the total recognized payment, the identity that should approximately hold is:
> allowed amount ≈ primary plan-paid amount + other/secondary-payer-paid amount + patient > responsibility (deductible + copay + coinsurance)
This is a diagnostic check, not a guarantee. It can legitimately fail to reconcile because of a missing coordination-of-benefits (COB) payment from a secondary payer, an unnetted reversal, balance billing above the allowed amount, a noncovered or withheld amount, an out-of-network claim where the identity does not apply in the same way, or because the file simply does not populate one of the components (common in encounter data — see below).
Plan-paid amount — the payer's own liability, and coordination of benefits
The plan-paid amount (paid amount, `paid_plan`) is the payer's own liability only — what the health plan itself actually disbursed, net of the patient's cost-sharing. It is the correct field for a payer-perspective analysis (e.g., a health plan's own budget-impact model) but understates total resource value whenever there is patient cost-sharing, and it becomes actively misleading under coordination of benefits (COB): when a patient has both a primary and a secondary payer (dual Medicare/Medicaid, a spouse's employer plan, a Medigap policy), the primary payer's claim shows only its own partial payment, and the secondary payer's claim shows the balance. Summing only the primary payer's `paid_plan` field silently drops everything the secondary payer covered. Allowed amount is the safer resource valuation exactly because it is a payer-contract quantity that does not depend on how many payers split the liability; if allowed amount is unavailable, both primary and secondary paid amounts must be identified and summed.
Patient responsibility — deductible, copay, coinsurance
Patient responsibility is decomposed into three distinct benefit-design mechanisms that behave differently across a plan year and must not be collapsed into one number without checking which ones the extract actually carries:
- Deductible: a fixed dollar amount the patient must pay for deductible-subject services before the plan begins sharing the cost of those services; many benefit designs exempt specific services (e.g., ACA-mandated preventive care) from the deductible entirely, so "before the plan pays anything" is not literally true across all service types on a plan. Deductibles typically reset each plan year, so early-year claims show a larger patient share than later-year claims for an otherwise identical service (a within-patient seasonality that can masquerade as a treatment effect if index dates cluster by calendar month).
- Copay: a fixed dollar amount per visit or fill, independent of the allowed amount.
- Coinsurance: a fixed percentage of the allowed amount (e.g., 20%), so it scales with the price of the service, unlike a copay. Some claims extracts carry only a single blended `patient_pay` field instead of the three components; when the three-way split is present, it is valuable for benefit-design and affordability research (e.g., high-deductible plan enrollment effects) but is not required simply to compute total cost — `plan_paid + patient_pay` (or, under COB, the sum across all payers plus patient pay) reconstructs the allowed amount either way.
Where these fields live in a claims extract, and how payer type changes what is populated
- Commercial FFS claims (Optum, MarketScan, HCCI, and similar vendors): All four fields are typically well populated — `billed`/`chg_amt`, `allowed`/`elig_amt`, `paid_plan`/`pay_amt`, and the cost-sharing components. Field names vary substantially by vendor; always confirm from the data dictionary whether "paid" means plan-paid-only or plan-paid-plus-COB.
- Medicare FFS (MedPAR, Carrier/Physician/Supplier, Outpatient SAF): Allowed and paid amounts are standardized payment-system outputs — MS-DRG base rate × wage index (plus outlier and disproportionate-share/indirect-medical-education adjustments) for inpatient, the RBRVS fee schedule (relative value units × conversion factor × geographic practice cost index, further adjusted for site of service, assignment, and modifiers) for professional services, APC for outpatient — which makes Medicare FFS one of the more internally consistent payers for multi-site or multi-year comparisons. This is not a single "same code, same price everywhere" formula, though: facility status (e.g., rural, critical-access, teaching), modifiers, case mix, outlier payments, and annual fee-schedule updates all still produce real, policy-relevant price variation on top of the geographic adjustment. Total charges (`clm_tot_chrg_amt`) are also present but remain a chargemaster artifact, not the payment. Medicare Part D Prescription Drug Event (PDE) records are structurally different from these administered-price outputs: a PDE reports several separate drug-payment components (negotiated ingredient cost, dispensing fee, sales tax, patient pay amount, low-income cost-sharing subsidy, catastrophic-coverage reinsurance, and gross covered drug cost) rather than a single allowed/paid pair, and point-of-sale PDE amounts do not reflect post-point-of-sale manufacturer rebates, so PDE fields must be unpacked component-by-component rather than mapped onto the billed/allowed/paid framework used for medical claims.
- Capitated Medicare Advantage (MA) and Medicaid managed-care (MCO) encounter data: This is the payer-type edge case that breaks a naive "just use paid amount" rule. CMS or the state pays the MA plan or Medicaid MCO a fixed per-member-per-month (PMPM) capitation regardless of how much care a member uses — but that capitation is paid to the plan, not necessarily to the individual provider: the plan's own downstream contract with a given provider may itself be capitation, FFS, salary, or another arrangement (Medicaid MCOs routinely use both FFS and fixed-periodic provider-payment arrangements), so a $0 or nominal amount on one encounter record does not by itself prove how, or whether, that specific provider was paid. The individual encounter record submitted for a member's ER visit or office visit is filed primarily for risk-adjustment and utilization reporting; MedPAC's review of MA encounter data notes that the "cost of the services provided" field is populated only when a capitated arrangement is not in place, so `paid_plan` on a capitated encounter is frequently $0, blank, or a small nominal/shadow-priced amount even though real resources were consumed, and should be treated as a non-valued or unreliable payment field unless the specific file's own documentation establishes what it means. A `paid_plan = 0` on an FFS claim usually means the claim was denied or fully patient-liable; a `paid_plan = 0` on an MA or Medicaid MCO encounter usually means the plan-level capitation arrangement makes the claim-level payment field structurally uninformative — the two zeros require opposite handling and must never be pooled without a `claim_type`/`encounter_type` flag distinguishing them (see medicare-ffs-ma-commercial-claims-differences-rwe). Allowed amounts on encounter data are also inconsistently populated; some MAOs shadow-price the encounter to what FFS Medicare would have allowed, others leave it blank or equal to billed charges.
- Medicaid FFS: Highly state-specific; T-MSIS and state MAX files vary in whether allowed and paid are separately reported or collapsed into one paid field, and in how thin state fee schedules make the allowed amount relative to commercial rates for the same code.
Negative and reversed amounts
Raw claims extracts are not a clean list of final adjudicated dollars; they are a transaction log. When a payer reprocesses a claim — correcting a diagnosis, adjusting a fee-schedule lookup, applying a late COB payment — the standard mechanism is to submit a full reversal (a line carrying the exact negative of every dollar field on the original line, zeroing it out) followed by a replacement line with the corrected amounts. An extract that is not netted by claim/service identifier before summing will double- or triple-count the service: naively filtering to "amounts greater than zero" drops the true reversal line (which is negative, so it looks like noise) but leaves both the original and the replacement positive lines in place, inflating billed, allowed, and paid totals for a single service. The correct approach is to net all dollar fields by grouping on the identifier that links original, reversal, and replacement lines together (the parent claim control number, or an original-claim-ID reference), summing rather than filtering by sign — see claim-adjustments-reversals-denials for the full reversal/void/replacement taxonomy this entry's netting logic depends on. The `-V` (void) and `-R#` (replacement) claim-ID suffixes used in this entry's worked example and reference code are a synthetic simplification for illustration only. Real sources signal claim lineage through vendor-specific fields — a frequency/type-of-bill digit, an original-claim-ID or ICN cross-reference field, an adjustment/void indicator, or a claim-status code — not a fixed string suffix on the claim identifier; any production netting logic must be built on the source's documented linkage field, not on string pattern-matching against `claim_id`.
Standardized pricing approaches for multi-payer studies
A study that pools claims across payers, states, or years cannot compare raw allowed amounts directly, because the same CPT/DRG is priced differently by every commercial contract and by Medicare's annually updated fee schedules. Three approaches appear in the literature:
- Fee-schedule repricing: Re-price every line to a single common fee schedule (most often the current-year Medicare RBRVS/OPPS/IPPS rates) using the procedure and diagnosis codes on the line, discarding the payer's own negotiated allowed amount entirely. This is the approach behind large employer-led hospital price-transparency initiatives that express commercial allowed amounts as a percentage of what Medicare would have paid for the same service (e.g., the RAND Hospital Price Transparency Study), yielding a comparable "Medicare-equivalent" price index across payers and geographies at the cost of erasing genuine, policy-relevant price variation.
- Actual allowed-amount price indexes (no repricing): Other multi-payer studies measure real-world negotiated-price variation directly from payers' own actual allowed amounts — defined as the sum of insurer and patient payments — without repricing to a common schedule, then build a risk- or service-mix-adjusted price index across markets. Cooper et al.'s Health Care Cost Institute (HCCI) analysis of privately insured claims is the canonical example: it documents multi-fold negotiated-price variation across hospitals for identical services using the payers' own actual allowed amounts, not Medicare-repriced amounts.
- Cost-to-charge ratios (CCR): For hospital-only data sources that report charges but not adjudicated allowed amounts (e.g., some state hospital discharge databases), a hospital-level cost-to-charge ratio — published annually by AHRQ's Healthcare Cost and Utilization Project (HCUP) from hospital Medicare cost reports — converts billed charges to an estimated true cost: `estimated_cost = billed_charge × CCR`. The published HCUP CCR files are hospital-level, not broken out by department or cost center, so this conversion is necessarily a facility-wide approximation; true within-hospital cost intensity varies by service line (e.g., the OR consumes resources very differently than pharmacy or room-and-board), and a hospital-wide CCR cannot capture that variation without a separate, internally sourced departmental cost-accounting file. None of the three approaches recovers the payer's original allowed amount exactly. Where actual adjudicated allowed amounts are available (most commercial, Medicare FFS, and many all-payer claims databases), using them directly is preferable to repricing or CCR conversion, both of which discard real transaction-price information; repricing and CCR conversion are explicit, pre-specifiable compromises reserved for sources that lack adjudicated allowed amounts, and must be reported as a methods choice, not treated as ground truth.
Pros, cons, and trade-offs — specific and comparative
- Allowed vs paid: Allowed is the correct system-cost denominator regardless of how many payers split the liability; paid is the correct payer-perspective numerator for a single named payer's own budget model. Using paid to represent total transaction-price spend understates cost by the entire patient-and-secondary-payer share, an understatement that grows mechanically across the plan year as members accumulate deductible credit.
- Allowed vs billed: Billed charges require no adjudication and are therefore almost always present, even on denied claims (though some encounter files and de-identified vendor extracts suppress or blank them), which makes them tempting as a "complete" field; but their magnitude is driven by chargemaster policy, not resource consumption, so any comparison across facilities, payers, or years using billed charges confounds transaction-price differences with unrelated chargemaster inflation differences.
- Fee-schedule repricing vs cost-to-charge ratios: Repricing preserves comparability across payers at the cost of erasing real negotiated-price variation (useful when the research question is "what would this cost under a common yardstick"); CCR preserves the facility's actual price level at the cost of only approximating true cost through an annual, hospital-level ratio that cannot resolve within-hospital service-line variation (useful when the research question is "what did this facility actually spend," and only charges are available).
When NOT to use — and when the wrong field is actively misleading or dangerous
- Reporting billed charges as "cost" without a validated conversion in any HTA, payer, or budget-impact submission. This is immediately flagged by any reviewer familiar with claims data and undermines the credibility of the entire analysis. (Billed charges remain appropriate when the research question is specifically about charge-based contracts, balance billing, uninsured/ out-of-network exposure, or as the direct input to an explicit CCR conversion.)
- Summing `paid_plan` alone when COB is present. Silently drops the secondary payer's share and understates total resource use, typically worst in dual-eligible and Medigap-covered cohorts.
- Treating `paid_plan = 0` on MA/Medicaid-MCO encounter data as "no cost incurred." The service happened; the claim-level payment field is structurally unreliable under a capitated plan-payment model regardless of how the plan actually compensated that specific provider. Pooling these zeros with genuinely denied FFS claims (also `paid_plan = 0`, but for the opposite reason) without a claim/encounter-type flag corrupts both the numerator and any denial-rate statistic.
- Summing raw claim lines without netting reversals and replacements. Produces inflated billed, allowed, and paid totals whenever a claim was corrected — a routine, not rare, event in adjudicated data.
- Comparing raw allowed amounts across payers or geographies without standardization. The same CPT code can legitimately carry a 3–5x difference in negotiated allowed amount across commercial payers in the same market; an unadjusted multi-payer pooled mean mixes true price variation with sampling variation in payer mix across arms.
Data-source operational depth
- Commercial claims (Optum, MarketScan, HCCI, payer-direct extracts): Field-name conventions vary by vendor; the practical first step on any new source is confirming, from the data dictionary, exactly which field is billed, which is allowed, which is paid-plan-only vs. paid-including-COB, and whether reversal/replacement lines are pre-netted or raw.
- Medicare FFS (MedPAR, Carrier, Outpatient SAF): Allowed and paid amounts are standardized-formula outputs and comparable across sites and time after annual fee-schedule updating and accounting for facility- and provider-specific adjustments; `clm_tot_chrg_amt` remains a chargemaster artifact and should not be used for cost. Part D PDE (Prescription Drug Event) records are structurally different — they report several separate drug-payment components (ingredient cost, dispensing fee, sales tax, patient pay amount, low-income subsidy, catastrophic reinsurance) rather than a single allowed/paid pair, and point-of-sale PDE amounts exclude post-point-of-sale manufacturer rebates.
- Medicare Advantage / Medicaid MCO encounter data: CMS/state capitation is paid to the plan, not necessarily to the individual provider on a claim-by-claim basis, so expect frequent `paid_plan = 0`, blank, or nominal shadow-priced amounts regardless of how the plan actually compensates that provider; allowed-amount completeness varies by MAO/MCO; always confirm the `claim_type`/`encounter_source` flag before pooling with FFS claims, and consider excluding or externally shadow-pricing capitated spans from a cost denominator entirely (see medicare-ffs-ma-commercial-claims-differences-rwe and healthcare-costs-pppm-pppy-pmpm for the person-time handling).
- State all-payer claims databases (APCDs): Most APCDs built on the APCD Council Common Data Layout capture charge, allowed, plan-paid, coordination-of-benefits, and patient-liability fields for adjudicated medical, pharmacy, and dental claims; confirm this against the specific state's implementation and completeness reporting rather than assuming it. Hospital discharge databases (e.g., HCUP State Inpatient/Emergency Department Databases), by contrast, frequently carry only billed charges at the facility level with no adjudicated allowed amount; a published, hospital-level cost-to-charge ratio (HCUP CCR files) is required before any cost interpretation of those sources.
- Benefit carve-outs: When pharmacy, behavioral health, or specialty drugs are carved out to a separate vendor (PBM, behavioral health administrator), the medical claims file's `paid_plan` will show $0 or simply omit those service lines entirely, not because the service was free but because payment is adjudicated and recorded in a different data source altogether — see benefit-carve-outs-medical-pharmacy-rwe.
Decision diagram
flowchart TD
Charge["Billed / submitted charge<br/>chargemaster list price<br/>(UB-04 FL47 / CMS-1500 charge field)"] --> Adj{"Adjudicated as FFS<br/>or capitated encounter?"}
Adj -->|"commercial / Medicare FFS"| Allowed["Allowed amount<br/>= plan-negotiated or fee-schedule price<br/>-> standard 'system cost'"]
Adj -->|"capitated MA / Medicaid MCO"| ZeroPaid["Encounter record<br/>paid_plan often $0 or nominal<br/>(plan paid via capitation; claim-level<br/>field does not show provider payment)"]
Allowed --> PlanPaid["Plan-paid amount<br/>payer's own liability"]
Allowed --> PatResp["Patient responsibility<br/>deductible + copay + coinsurance"]
PatResp --> COB{"Secondary payer / COB?"}
COB -->|"yes"| SecPay["Secondary plan-paid amount<br/>reduces true patient out-of-pocket"]
COB -->|"no"| OOP["Patient out-of-pocket"]
ZeroPaid --> Shadow["Shadow-price to FFS-equivalent<br/>or exclude from cost denominator"]flowchart LR
Raw["Raw claim lines<br/>original + reversal + replacement"] --> Root["Group by root claim /<br/>service-event identifier"]
Root --> Net["Net billed, allowed, paid_plan,<br/>other_payer_paid, copay, coinsurance,<br/>deductible (sum, never filter by sign)"]
Net --> Check{"allowed ≈ paid_plan +<br/>other_payer_paid +<br/>patient_responsibility?"}
Check -->|"yes"| Use["Use allowed for system cost;<br/>paid for payer perspective"]
Check -->|"no"| Investigate["Investigate: missing COB payment,<br/>un-netted reversal, or<br/>encounter-data $0 paid"]
Use --> Reprice{"Multi-payer /<br/>multi-facility study?"}
Reprice -->|"yes"| Standardize["Fee-schedule repricing<br/>or cost-to-charge ratio"]
Reprice -->|"no"| Study["Use netted allowed / paid<br/>directly in cost analysis"]Worked example
Scenario
A researcher pulls three raw claim-line records for a single emergency-room visit (patient 7042, claim ER5001) from a commercial claims extract: the original submission, a full reversal the payer issued after re-adjudicating the diagnosis code, and a corrected replacement line. She needs to compute the correct billed, allowed, plan-paid, and patient-responsibility totals for this one visit, and to see what happens if she makes the common mistake of filtering out only rows with a negative dollar amount instead of properly netting the three lines together.
Dataset
Three raw claim-line rows for the same ER visit, as they would appear in a claims extract before any cleaning. Amounts are in US dollars.
| claim_id | line_type | billed | allowed | paid_plan | copay | coinsurance | deductible |
|---|---|---|---|---|---|---|---|
| ER5001 | original | 12000 | 1850 | 1480 | 50 | 220 | 100 |
| ER5001-V | reversal | -12000 | -1850 | -1480 | -50 | -220 | -100 |
| ER5001-R1 | replacement | 12000 | 1900 | 1520 | 50 | 230 | 100 |
Steps
Result
Correctly netted, the ER visit's values are: billed $12,000 (uninformative for cost), allowed $1,900 (the transaction-price-based system-cost figure for this single-payer example), plan-paid $1,520, and patient responsibility $380 ($50 copay + $230 coinsurance + $100 deductible), with plan-paid + patient responsibility reconciling exactly to allowed. Filtering by sign instead of netting by claim identifier doubles the apparent cost to $3,750 allowed / $3,000 paid. Using the billed charge as a cost proxy would overstate the transaction-price-based ($1,900) system cost by 6.3x in this illustrative example — a charge-to-allowed statistic distinct from, though qualitatively consistent with, the published charge-to-cost literature (see steps above).
Trade-offs
Runnable example
Utility functions for the three core payment-anatomy operations demonstrated in the worked example: (1) netting billed/allowed/paid/patient-responsibility fields across original, reversal, and replacement claim lines, (2) repricing billed charges to an estimated cost using a cost-to-charge ratio when no allowed...
import re
import pandas as pd
DOLLAR_COLS = ["billed", "allowed", "paid_plan", "other_payer_paid", "copay", "coinsurance", "deductible"]
PATIENT_COLS = ["copay", "coinsurance", "deductible"]
# other_payer_paid is OPTIONAL: populated only on claims with a coordination-
# of-benefits (COB) secondary payer. When the source lacks this column
# entirely, net_payment_fields() adds it as all-missing and the
# reconciliation check below treats it as $0, matching a single-payer claim.
# ------------------------------------------------------------------
# 1. NET REVERSAL/REPLACEMENT LINES BY ROOT CLAIM/SERVICE-EVENT ID
# Input: raw claim-line rows, one per original/reversal/replacement
#
# NOTE: the '-V'/'-R#' claim_id suffix convention below is a SYNTHETIC
# simplification for this worked example only. Real sources signal
# claim lineage through vendor-specific fields (a frequency/type-of-
# bill digit, an original-claim-ID/ICN cross-reference field, an
# adjustment/void indicator, or a claim-status code) -- never assume a
# fixed string suffix on claim_id; derive root_claim_id from the
# source's documented linkage field instead. See
# claim-adjustments-reversals-denials for the full transaction taxonomy.
# ------------------------------------------------------------------
def _root_claim_id(claim_id: str) -> str:
"""Strip '-V' (void/reversal) and '-R#' (replacement) suffixes to
find the parent claim/service-event identifier the lines share.
SYNTHETIC EXAMPLE CONVENTION ONLY -- see module note above."""
return re.sub(r"-(V|R\d+)$", "", str(claim_id))
def net_payment_fields(lines: pd.DataFrame) -> pd.DataFrame:
"""
Net billed/allowed/paid/other-payer-paid/patient-responsibility dollar
fields to one row per service event, correctly absorbing full-reversal
lines (exact negatives) and replacement lines carried as separate rows.
Input columns: claim_id, billed, allowed, paid_plan, copay,
coinsurance, deductible, and optionally
other_payer_paid (a secondary payer's COB payment;
added as all-missing if the source omits it entirely).
Returns one row per root claim/service event with netted dollar
fields, total patient_responsibility, a components_complete flag,
and a reconciliation check (allowed vs. paid_plan +
other_payer_paid + patient_responsibility) that is left missing (NA),
not silently marked True, when a required component is entirely
missing for that service event -- unknown data must never be
reported as a verified reconciliation. other_payer_paid itself is
NOT a required component (most claims have no secondary payer): when
entirely absent for a service event it is treated as $0 in the
reconciliation check, never as a reason to mark the check unknown.
"""
lines = lines.copy()
if "other_payer_paid" not in lines.columns:
lines["other_payer_paid"] = pd.NA
lines["root_claim_id"] = lines["claim_id"].apply(_root_claim_id)
# min_count=1: an all-missing group sums to NaN, not 0, so a service
# event with no rows carrying a given field stays flagged as unknown.
net = lines.groupby("root_claim_id")[DOLLAR_COLS].agg(
lambda s: s.sum(min_count=1)
).reset_index()
required_cols = PATIENT_COLS + ["allowed", "paid_plan"]
net["components_complete"] = net[required_cols].notna().all(axis=1)
net["patient_responsibility"] = net[PATIENT_COLS].sum(axis=1, min_count=1)
net["reconciles"] = pd.NA
complete = net["components_complete"]
other_payer_paid_amt = net["other_payer_paid"].fillna(0)
net.loc[complete, "reconciles"] = (
(net.loc[complete, "paid_plan"] + other_payer_paid_amt.loc[complete]
+ net.loc[complete, "patient_responsibility"]
- net.loc[complete, "allowed"]).abs() < 0.01
)
net["charge_to_allowed_ratio"] = net["billed"] / net["allowed"].replace(0, pd.NA)
return net
# ------------------------------------------------------------------
# 2. COST-TO-CHARGE RATIO REPRICING (hospital-only charge data)
# ------------------------------------------------------------------
def apply_cost_to_charge_ratio(billed_charges: pd.Series, ccr: float) -> pd.Series:
"""
Reprice billed charges to an estimated cost using a hospital-level
cost-to-charge ratio (e.g., from HCUP CCR files -- these are
published at the hospital level, not department/cost-center level).
Use only when no adjudicated allowed amount exists in the source,
and treat the result as a facility-wide approximation, not a
service-line-precise cost.
"""
return billed_charges * ccr
# ------------------------------------------------------------------
# 3. FLAG CAPITATED $0-PAID ENCOUNTERS (MA / Medicaid MCO)
# ------------------------------------------------------------------
CAPITATED_ENCOUNTER_TYPES = {"ma_encounter", "medicaid_mco_encounter"}
def flag_capitated_zero_paid(claims: pd.DataFrame) -> pd.Series:
"""
Flag capitated encounter lines with paid_plan == 0 as 'no valid cost
numerator' rather than 'free care', so a downstream cost pipeline can
exclude (or externally shadow-price) them instead of biasing a pooled
FFS + MA cost mean toward zero.
"""
return claims["claim_type"].isin(CAPITATED_ENCOUNTER_TYPES) & (claims["paid_plan"] == 0)
# ------------------------------------------------------------------
# EXAMPLE USAGE (mirrors the worked example)
# ------------------------------------------------------------------
if __name__ == "__main__":
raw_lines = pd.DataFrame({
"claim_id": ["ER5001", "ER5001-V", "ER5001-R1"],
"billed": [12000, -12000, 12000],
"allowed": [1850, -1850, 1900],
"paid_plan": [1480, -1480, 1520],
"copay": [50, -50, 50],
"coinsurance": [220, -220, 230],
"deductible": [100, -100, 100],
})
netted = net_payment_fields(raw_lines)
print(netted[["root_claim_id", "billed", "allowed", "paid_plan", "other_payer_paid",
"patient_responsibility", "reconciles", "charge_to_allowed_ratio"]])
# Expected: billed=12000, allowed=1900, paid_plan=1520,
# other_payer_paid=NaN (absent from source -> treated as $0
# in the reconciliation check only), patient_responsibility=380,
# reconciles=True, ratio=6.32R functions for the same three payment-anatomy operations: netting original/reversal/replacement claim lines, cost-to-charge ratio repricing, and flagging capitated $0-paid encounters. Mirrors the Python implementation and worked example exactly.
library(data.table)
DOLLAR_COLS <- c("billed", "allowed", "paid_plan", "other_payer_paid", "copay", "coinsurance", "deductible")
PATIENT_COLS <- c("copay", "coinsurance", "deductible")
# other_payer_paid is OPTIONAL: populated only on COB claims with a
# secondary payer. net_payment_fields() adds it as NA if the source
# lacks the column entirely, and treats it as $0 in the reconciliation
# check (never as a reason to mark the check unknown).
# ------------------------------------------------------------------
# 1. NET REVERSAL/REPLACEMENT LINES BY ROOT CLAIM ID
#
# NOTE: the '-V'/'-R#' claim_id suffix convention below is a SYNTHETIC
# simplification for this worked example only. Real sources signal
# claim lineage through vendor-specific fields (a frequency/type-of-
# bill digit, an original-claim-ID/ICN cross-reference field, an
# adjustment/void indicator, or a claim-status code) -- derive
# root_claim_id from the source's documented linkage field, never from
# a fixed string suffix. See claim-adjustments-reversals-denials for
# the full transaction taxonomy.
# ------------------------------------------------------------------
root_claim_id <- function(claim_id) {
sub("-(V|R[0-9]+)$", "", claim_id)
}
net_payment_fields <- function(lines) {
# R's sum() propagates NA by default (unlike pandas' skipna=TRUE), so an
# all-missing dollar column for a service event already stays NA here --
# components_complete makes that behavior explicit rather than implicit.
dt <- as.data.table(lines)
if (!"other_payer_paid" %in% names(dt)) {
dt[, other_payer_paid := NA_real_]
}
dt[, root_claim_id := root_claim_id(claim_id)]
net <- dt[, lapply(.SD, sum), by = root_claim_id, .SDcols = DOLLAR_COLS]
net[, components_complete := !is.na(allowed) & !is.na(paid_plan) &
!is.na(copay) & !is.na(coinsurance) & !is.na(deductible)]
# other_payer_paid is NOT required for components_complete -- most
# claims have no secondary payer. When entirely absent for a service
# event it is treated as $0 in the reconciliation check below.
net[, patient_responsibility := copay + coinsurance + deductible]
net[, other_payer_paid_amt := ifelse(is.na(other_payer_paid), 0, other_payer_paid)]
net[, reconciles := ifelse(components_complete,
abs(paid_plan + other_payer_paid_amt + patient_responsibility - allowed) < 0.01,
NA)]
net[, charge_to_allowed_ratio := billed / allowed]
net
}
# ------------------------------------------------------------------
# 2. COST-TO-CHARGE RATIO REPRICING (hospital-level HCUP CCR files --
# not department/cost-center-level; treat as a facility-wide estimate)
# ------------------------------------------------------------------
apply_cost_to_charge_ratio <- function(billed_charges, ccr) {
billed_charges * ccr
}
# ------------------------------------------------------------------
# 3. FLAG CAPITATED $0-PAID ENCOUNTERS
# ------------------------------------------------------------------
CAPITATED_ENCOUNTER_TYPES <- c("ma_encounter", "medicaid_mco_encounter")
flag_capitated_zero_paid <- function(claims) {
claims$claim_type %in% CAPITATED_ENCOUNTER_TYPES & claims$paid_plan == 0
}
# ------------------------------------------------------------------
# EXAMPLE USAGE
# ------------------------------------------------------------------
raw_lines <- data.table(
claim_id = c("ER5001", "ER5001-V", "ER5001-R1"),
billed = c(12000, -12000, 12000),
allowed = c(1850, -1850, 1900),
paid_plan = c(1480, -1480, 1520),
copay = c(50, -50, 50),
coinsurance = c(220, -220, 230),
deductible = c(100, -100, 100)
)
netted <- net_payment_fields(raw_lines)
print(netted[, .(root_claim_id, billed, allowed, paid_plan, other_payer_paid,
patient_responsibility, components_complete, reconciles,
charge_to_allowed_ratio)])
# Expected: billed=12000, allowed=1900, paid_plan=1520,
# other_payer_paid=NA (absent from source -> treated as $0 in
# the reconciliation check only), patient_responsibility=380,
# components_complete=TRUE, reconciles=TRUE, ratio=6.32PROC SQL netting of raw claim lines to one row per service event (summing rather than filtering by sign), followed by a cost-to-charge ratio repricing step for a hospital-charge-only comparison file. Mirrors the Python/R worked-example logic.
/* 1. Derive root claim/service-event id by stripping a trailing -V (void) or
-R# (replacement) suffix, ANCHORED to the end of the string via PRXMATCH
so an embedded '-V' or '-R' elsewhere in the identifier is never matched.
NOTE: this -V/-R# suffix convention is a SYNTHETIC simplification for
this worked example only. Real sources signal claim lineage through
vendor-specific fields (a frequency/type-of-bill digit, an original-
claim-ID/ICN cross-reference field, an adjustment/void indicator, or a
claim-status code) -- derive root_claim_id from the source's documented
linkage field in production, never from a string-suffix heuristic. See
claim-adjustments-reversals-denials for the full transaction taxonomy. */
data claims_prep;
set work.raw_claim_lines;
length root_claim_id $20;
retain suffix_re;
if _n_ = 1 then suffix_re = prxparse('/-(V|R\d+)$/');
pos = prxmatch(suffix_re, trim(claim_id));
if pos > 0 then root_claim_id = substr(claim_id, 1, pos - 1);
else root_claim_id = claim_id;
drop pos;
/* other_payer_paid is OPTIONAL -- populated only on COB claims with a
secondary payer. This self-assignment is the standard SAS idiom for
safely referencing a variable that may not exist on the input
table: if work.raw_claim_lines carries the column, its values pass
through unchanged; if it does not, SAS creates the variable here as
missing for every row (a harmless "uninitialized variable" note in
the log, not an error) so it nets like any other dollar field below. */
other_payer_paid = other_payer_paid;
run;
/* 2. Net all dollar fields by root_claim_id (sum, not filter-by-sign), and
flag whether every required component was non-missing on every raw row
contributing to the group -- SAS's SUM() silently skips missing values,
so without this flag a service event with wholly missing patient-
responsibility fields would falsely appear to reconcile at 0.
other_payer_paid is netted but deliberately excluded from
components_complete: most claims have no secondary payer, so an
entirely missing other_payer_paid must not block reconciliation. */
proc sql;
create table claims_netted as
select root_claim_id,
sum(billed) as billed,
sum(allowed) as allowed,
sum(paid_plan) as paid_plan,
sum(other_payer_paid) as other_payer_paid,
sum(copay + coinsurance + deductible) as patient_responsibility,
(count(allowed) = count(*) and count(paid_plan) = count(*) and
count(copay) = count(*) and count(coinsurance) = count(*) and
count(deductible) = count(*)) as components_complete
from claims_prep
group by root_claim_id;
quit;
/* 3. Reconciliation check: paid_plan + other_payer_paid + patient_responsibility
should equal allowed -- only evaluated when components_complete = 1; left
missing otherwise rather than defaulting to a false "reconciles".
other_payer_paid defaults to 0 via COALESCE when entirely absent for a
service event (no secondary payer on record), not treated as unknown.
Also derive the illustrative charge-to-allowed ratio (distinct from a
published charge-to-cost statistic -- see long_description). */
data claims_checked;
set claims_netted;
other_payer_paid_amt = coalesce(other_payer_paid, 0);
if components_complete = 1 then
reconciles = (abs(paid_plan + other_payer_paid_amt + patient_responsibility - allowed) < 0.01);
else reconciles = .;
if allowed ne 0 then charge_to_allowed_ratio = billed / allowed;
run;
/* 4. Cost-to-charge ratio repricing for a hospital-charge-only comparison
source (use only when no adjudicated allowed amount exists there). The
HCUP CCR files this ratio is typically drawn from are hospital-level,
not department/cost-center-level, so estimated_cost is a facility-wide
approximation, not a service-line-precise cost. */
data hospital_est_cost;
set work.hospital_charges_only;
estimated_cost = billed_charge * cost_to_charge_ratio;
run;Citations
- [1]Tyree PT, Lind BK, Lafferty WE. Challenges of using medical insurance claims data for utilization analysis. American Journal of Medical Quality. 2006;21(4):269-275.
- [2]Berger ML, Mamdani M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part I. Value in Health. 2009;12(8):1044-1052.
- [3]Centers for Medicare & Medicaid Services (CMS). Glossary of Health Coverage and Medical Terms (Uniform Glossary, required under 45 CFR 147.200). Baltimore, MD: CMS; 2023.
- [5]Cooper Z, Craig SV, Gaynor M, Van Reenen J. The price ain't right? Hospital prices and health spending on the privately insured. Quarterly Journal of Economics. 2019;134(1):51-107.
- [6]Whaley CM, Kerber R, Wang D, Kofner A, Briscombe B. Prices Paid to Hospitals by Private Health Plans: Findings from Round 5.1 of an Employer-Led Transparency Initiative. Santa Monica, CA: RAND Corporation; 2024.
- [7]Healthcare Cost and Utilization Project (HCUP). HCUP Cost-to-Charge Ratio (CCR) Files. Rockville, MD: Agency for Healthcare Research and Quality; accessed 2026.
- [8]Medicare Payment Advisory Commission (MedPAC). Ensuring the accuracy and completeness of Medicare Advantage encounter data. In: Report to the Congress: Medicare and the Health Care Delivery System, Chapter 7. Washington, DC: MedPAC; June 2019.
- [9]Medicaid and CHIP Payment and Access Commission (MACPAC). Medicaid Managed Care Payment. Washington, DC: MACPAC; accessed 2026.
- [10]APCD Council. APCD Common Data Layout (APCD-CDL): Medical Claims File Specification. Denver, CO: National Association of Health Data Organizations / APCD Council; accessed 2026.