UB-04 / 837I Institutional Claim Fields

In plain language

A UB-04 is the billing form that hospitals and other care facilities — not doctors — send to insurance when a patient stays overnight, visits the emergency room, or receives outpatient services. Each line on the form carries a structured code: a four-digit "type of bill" that tells the insurer whether the stay was inpatient or outpatient, a two-digit discharge status saying where the patient went when they left (home, another hospital, or the morgue), and a "present-on-admission" flag on every diagnosis that tells whether the patient arrived with that condition or developed it during the stay. Researchers use these codes to build hospitalization episodes, count in-hospital deaths, separate complications from preexisting conditions, and measure facility costs — but the raw data require careful cleaning to handle interim bills, replacement claims, and the critical difference between billed charges and actual costs.

What the UB-04 and 837I are, and who files them

The UB-04 (also called the CMS-1450) is the paper claim form maintained by the National Uniform Billing Committee (NUBC), whose normative code sets are copyrighted by the American Hospital Association. Its electronic counterpart is the 837I (Institutional) X12 transaction set, the digital file that clearinghouses and payers actually process. Every field described here is documented in the publicly available CMS Medicare Claims Processing Manual (Pub 100-04, Chapter 25), which is the source for the field semantics used throughout this entry; the complete NUBC data specifications are a licensed product.

Institutional claims are filed by hospitals (inpatient and outpatient departments including the ED), skilled nursing facilities (SNFs), home health agencies (HHAs), hospice programs, hospital-based dialysis units, and outpatient rehabilitation facilities. The counterpart billing instrument for physician and professional services is the CMS-1500 / 837P; the two forms are complementary, not redundant — most hospital admissions generate both a facility UB-04 claim (the institution's costs) and one or more professional 837P claims (physician fees). Research databases derived from adjudicated Medicare or commercial claims typically separate these into an inpatient/outpatient facility file (MedPAR, outpatient SAF, or equivalent) and a professional/carrier file.

Field anatomy by Form Locator (FL)

The UB-04 organizes its data elements by Form Locator number (FL 1 through FL 81). Understanding the research significance of each is the key to correct institutional claims analysis.

FL4 — Type of Bill (TOB): The single most important routing and classification field on the form. TOB is a four-digit code with a leading zero, where each position encodes a distinct dimension: - Digit 1 (always 0, the leading placeholder) - Digit 2 (facility type): 1 = hospital, 2 = SNF, 3 = home health, 4 = religious/non-medical, 5 = intermediate care, 6 = intermediate care-mentally retarded, 7 = clinic, 8 = special facility - Digit 3 (bill classification): 1 = inpatient Part A, 2 = inpatient Part B, 3 = outpatient, 4 = other Part B, 5 = intermediate care, 6 = intermediate care-mentally retarded, 7 = subacute inpatient, 8 = swing bed - Digit 4 (frequency/sequence): 1 = admit-through-discharge (the normal, complete bill), 2 = interim-first claim, 3 = interim-continuing, 4 = interim-last, 7 = replacement of a prior claim, 8 = void/cancel of a prior claim

The TOB frequency digit is the most important deduplication signal in institutional claims

A standard inpatient admission produces TOB 0111 (hospital inpatient, admit-through-discharge). If the stay spans a billing period boundary or interim billing is triggered, the hospital may submit a series of interim claims (frequency 2, 3, 4) followed by the final admit-through-discharge bill (frequency 1). Research extracts that do not roll up these interim bills will double- or triple-count the same admission, inflating utilization counts and visit-day totals. Replacement claims (frequency digit 7) supersede a previously submitted bill; voided claims (digit 8) cancel it entirely. A clean claims extract must: (a) exclude interim claims (frequency 2, 3) or explicitly roll them to the final bill, (b) apply replacements (keep the highest-sequence replacement, discard the original), and (c) exclude voids. Research databases such as MedPAR already implement this deduplication for inpatient stays; outpatient and SNF files typically do not, requiring analyst-level claim adjudication.

Beyond deduplication, TOB is the field from which research databases derive care setting — the distinction between inpatient and outpatient. TOB 011x indicates inpatient hospital; 013x indicates outpatient hospital (including the emergency department); 021x indicates SNF inpatient; 032x-033x indicates home health. A classifier that relies only on the claim type file or on ICD-10-PCS codes without confirming the TOB will misassign setting in edge cases.

FL6 — Statement Covers Period (From/Through dates): The calendar span that the claim covers — the from date (admission date for inpatient, service start for outpatient) and the through date (discharge date or last service date). For research, the through date determines when the episode ends; the from date is the service initiation. These dates are distinct from the claim receipt date and the adjudication date, both of which can lag weeks behind the service event. Adjudication lag — the gap between service delivery and final payment — means the final months of any claims extract are systematically under-reported; analysts should apply a 90-day run-out window before treating recent utilization as complete.

FL14 — Priority/Type of Admission and FL15 — Point of Origin for Admission: FL14 codes the urgency of the admission: 1 = emergency, 2 = urgent, 3 = elective, 4 = newborn, 5 = trauma. FL15 codes the source of the admission: 1 = non-healthcare facility (community), 2 = clinic, 4 = transfer from a hospital, 5 = transfer from a SNF, 6 = transfer from another healthcare facility, 8 = court/law enforcement. Together, FL14 and FL15 are used in real-world evidence for emergency-admission phenotypes, transfer chain construction, and social determinants of health (SDOH) studies that flag admissions originating from institutional settings such as long-term care. A limitation: these fields are completed at admission and can reflect pre-admission coding conventions rather than clinical reality, and their completeness varies by payer and provider.

FL17 — Patient Discharge Status: A two-digit code indicating the patient's disposition at the end of the stay, as documented in CMS guidance: 01 = discharged to home or self-care (routine discharge), 02 = discharged/transferred to a short-term general hospital for inpatient care, 03 = discharged to skilled nursing facility, 04 = discharged to intermediate care facility, 05 = discharged to another type of institution, 06 = discharged to home under care of organized home health service, 07 = left against medical advice, 20 = expired (death during stay), 30 = still patient (not yet discharged), 43 = discharged to a federal hospital, 50–57 = various hospice dispositions, 61–65 = swing-bed transfers and rehabilitation.

The research significance of discharge status is threefold: - In-hospital mortality: status 20 is the standard claims-based ascertainment of death during the index hospitalization, used throughout comparative effectiveness and safety research as one component of a composite mortality outcome. - Transfer chain construction: status 02 (transfer to another acute hospital) triggers a multi-claim linkage problem. When two acute hospital claims are connected by a status-02 discharge and the receiving hospital's admission date matches the transfer date, they should be treated as a single episode, not two readmissions. Failure to merge transfer chains inflates readmission rates and distorts 30-day outcomes. - Still-patient exclusion: status 30 means the patient was still hospitalized when the billing period closed; these patients must be excluded from readmission rate denominators and 30-day mortality calculations because follow-up is undefined. Status-30 claims are more common in long stays and in hospital-to-SNF or hospital-to-rehabilitation transitions where the formal discharge is delayed.

FL18–28 — Condition Codes: Up to eleven two-digit codes that communicate special billing conditions — for example, code 04 (information only, not for Medicare adjudication), 07 (treatment of non-terminal condition for hospice patient), D9 (any other special condition). Researchers rarely use condition codes directly in outcome or exposure algorithms, but they carry claim-processing signals that can explain anomalies in adjudicated files: a condition code signaling a partial episode or a carve-out arrangement can produce a claim that looks like utilization but does not reflect a complete billable encounter.

FL31–36 — Occurrence Codes and Dates: Paired code-and-date fields that record discrete events associated with the claim — for example, code 01 (accident/medical coverage), 11 (onset of symptoms), 17 (date outpatient occupational therapy plan established), A1 (birth date of insured). For RWE, occurrence codes 01 and 02 (auto accident and no-fault accident) are used in injury mechanism studies and trauma-cost analyses; code 11 (symptom onset) provides a claim-level date that is closer to true disease onset than the service date. These fields are often under-populated in commercial data.

FL39–41 — Value Codes and Amounts: Paired numeric codes and dollar amounts encoding episode-level financial and clinical quantities — for example, code 50 (physical therapy visits), 80 (covered days), A1 (deductible payer A). Researchers occasionally use covered-day value codes to derive SNF covered days (for benefit-period analysis) or to reconcile total charges.

FL42–49 — The Revenue Code Line Level (the detail or line level of the claim): This is where the encounter's services are itemized. Each line contains: - FL42 Revenue code: A four-digit code for the service category (e.g., 0100 = all-inclusive rate, 0120 = room and board semi-private, 0260 = IV therapy, 0450 = emergency room, 0490 = ambulatory surgery). Revenue codes form the institutional counterpart to CPT on the professional claim; they are how cost-center-level data are organized and are the mechanism for decomposing total claim cost by type of service. - FL44 — HCPCS/Rates: The procedure code at the line level, populated for outpatient claims with CPT-4 or HCPCS Level II codes. This is where outpatient procedure coding lives on an institutional claim — not in a separate procedure field as on the CMS-1500. For outpatient hospital claims, FL44 is the field that carries the CPT code used to identify a procedure-based phenotype (e.g., colonoscopy CPT 45378). For inpatient claims, the line-level HCPCS field is usually not populated because procedures are reported via ICD-10-PCS codes at the claim header. - FL46 — Units of Service: The quantity of the service on that line — room-and-board days, number of therapy sessions, units of a drug administered. Essential for utilization measurement (days, visits, doses). - FL47 — Total Charges: The facility's billed charges for that revenue line. Charges are not cost and not payment. The costing trap: chargemaster billed amounts overstate the actual resource cost by a factor that varies enormously by facility, service line, payer contract, and calendar year. Research that uses FL47 charges as a cost measure without applying a cost-to-charge ratio (CCR) or using the adjudicated allowed/paid amount is methodologically indefensible to a payer or HTA reviewer. The correct valuation for "cost to the system" is the allowed amount from the adjudicated claim; the correct payer perspective is the plan-paid amount.

FL67 — Principal Diagnosis + POA Indicator and FL67A–Q — Secondary Diagnoses + POA Indicators: These fields carry the ICD-10-CM diagnosis codes for the hospitalization plus the Present-on-Admission (POA) indicator for each diagnosis. POA was mandated for Medicare FFS inpatient claims starting October 1, 2007; its presence is what makes institutional claims from that date forward uniquely powerful for outcome research.

POA indicator values (per CMS documentation): - Y = Yes, condition was present on admission - N = No, condition was not present on admission (arose during the inpatient stay) - U = Unknown whether condition was present on admission - W = Clinically undetermined (provider unable to determine) - 1 = Exempt from POA reporting (certain ICD-10-CM codes are exempt)

The research value of POA: - Complication vs comorbidity distinction: A secondary diagnosis coded POA = N arose after admission and may represent a hospital-acquired complication (e.g., a catheter-associated UTI, a pressure injury, a deep vein thrombosis). A diagnosis coded POA = Y was a preexisting comorbidity. Without POA, it is impossible to determine from the billing record whether a secondary diagnosis code reflects patient severity at admission or an adverse event that occurred during the stay — a critical distinction for quality measurement, adverse-event outcomes algorithms, and risk adjustment. - Elixhauser and Charlson comorbidity scoring with POA: The standard approach for hospital-level risk adjustment applies the Elixhauser or Charlson comorbidity index only to diagnoses coded POA = Y (or exempt), excluding POA = N conditions that arose in-hospital and thus cannot be preexisting comorbidities. Applying comorbidity weights to all secondary diagnoses regardless of POA inflates the apparent comorbidity burden by including complications as covariates, biasing risk-adjusted outcomes. - Patient Safety Indicators (PSIs) and Hospital-Acquired Conditions (HACs): CMS uses POA to define HAC categories — conditions that are reimbursed differently (at lower rates) if they were not present on admission. AHRQ PSI algorithms rely on POA = N for their numerators. POA indicator validity has been studied and found generally reliable for major diagnoses but less consistent for secondary and minor conditions.

FL69 — Admitting Diagnosis: The ICD-10-CM code for what the patient was suspected to have at the time of admission — before workup, testing, and clinical evolution. FL69 often differs from the principal diagnosis (FL67), which is determined after the stay as the condition chiefly responsible for the admission. The admitting diagnosis is valuable in RWE for constructing unscheduled admission phenotypes (e.g., distinguishing a planned elective surgery admission from an acute unscheduled hospitalization) and in studies of emergency department-to-inpatient transitions. The gap between admitting diagnosis and principal diagnosis also serves as a signal of diagnostic ambiguity on admission.

FL70 — Patient Reason for Visit: Populated on unscheduled outpatient and emergency department claims; captures the patient-reported or triage-assigned chief complaint at the time of the ED or outpatient visit. Less consistently coded than FL67 but provides a pre-workup perspective distinct from the principal diagnosis assigned after the encounter.

FL71 — PPS/DRG: For Medicare inpatient claims under the Inpatient Prospective Payment System (IPPS), this field carries the assigned Medicare Severity Diagnosis Related Group (MS-DRG) used to determine the payment. The MS-DRG is a case-mix and severity classifier — it rolls up principal diagnosis, secondary diagnoses, procedures, and discharge status into a single reimbursement category. MS-DRG is used in RWE as a parsimonious case-mix adjuster (avoiding the full ICD-10-CM code matrix) and to define clinically coherent admission strata.

FL72 — External Cause of Injury (ECI) / E-code: ICD-10-CM external cause codes that describe the mechanism of an injury (e.g., fall, motor vehicle accident, assault). Used in trauma epidemiology, injury-mechanism cost studies, and SDOH analyses. These codes are under-reported and have variable completeness across payers.

FL74 — Principal Procedure + Date and FL74a–e — Other Procedures + Dates: ICD-10-PCS procedure codes and their dates for inpatient claims. Up to six procedures per claim (one principal, five additional). Unlike CPT on the professional claim, ICD-10-PCS is the coding system for inpatient hospital procedures. Procedure dates allow sequencing of surgical and procedural events within a stay — critical for surgical complication studies and time-to-procedure analyses.

FL76–79 — Attending, Operating, and Other Physician NPIs: National Provider Identifiers for the attending physician (FL76), operating physician (FL77), and up to two other significant providers (FL78, FL79). These fields are the institutional claim's mechanism for provider attribution — linking a hospital stay to the clinician responsible for care. Used in care variation studies, physician practice pattern analyses, and multi-payer attribution algorithms. Note that the operating physician NPI (FL77) is populated on claims with a surgical procedure and may differ from the attending — important for studies that need to distinguish surgeon from hospitalist.

The deduplication and claim-adjustment problem in detail

The single most common technical error in institutional claims analysis is failing to handle interim billing, replacement claims, and late charges before constructing utilization measures.

An inpatient stay longer than a monthly billing cycle (e.g., a 45-day ICU admission) will generate: - One or more interim-first (frequency 2) and interim-continuing (frequency 3) claims covering sub-periods of the stay, each with a through date before the actual discharge - A final admit-through-discharge claim (frequency 1) covering the full stay

If an analyst counts each of these as a separate hospitalization, a single 45-day stay becomes three admissions in the denominator, inflating hospitalization rates by a factor of three. The correct approach is to deduplicate by rolling up to the admit-through-discharge claim or, if using a preprocessed file like MedPAR, to verify that the preprocessing already performed this step.

Replacement claims (frequency 7) arise when a hospital corrects a submitted bill — updating diagnosis codes, charges, or dates. The replacement supersedes the original; an extract that retains both will double-count the claim and, if the diagnosis codes changed, carry inconsistent code sets for the same admission. The standard approach is to keep only the highest-sequence replacement for each original claim control number.

Late charge claims are a related problem: a small additional charge (e.g., a lab result that arrived after the discharge bill was submitted) may be filed as a new claim referencing the original admission. These late-charge claims, if not merged back to the parent claim, inflate admission counts.

Pros, cons, and trade-offs — specific and comparative

• vs the professional claim (CMS-1500 / 837P): The institutional claim captures the full facility

• vs ICD-10-PCS procedure codes alone: ICD-10-PCS procedure codes on the institutional claim

• vs EHR-derived discharge data: The UB-04 discharge status (FL17) reflects the billing-finalized

• Charges (FL47) vs allowed/paid amounts: Billed charges on the institutional claim are the

When to use

Use the UB-04/837I institutional claim fields as the primary data source whenever the research question requires: (1) care-setting determination (inpatient vs outpatient vs SNF vs home health), (2) episode length and boundaries (from/through dates plus discharge status), (3) in-hospital death ascertainment (discharge status 20), (4) complication vs comorbidity distinction (POA indicators), (5) transfer chain construction (discharge status 02 linked to receiving hospital admission), (6) facility-level cost decomposition by service line (revenue code × allowed amount), or (7) inpatient procedure coding (ICD-10-PCS via FL74). Institutional claims are the correct and complete data source for any outcome or utilization variable that is anchored to a hospital stay, SNF episode, or home health certification period.

When NOT to use — and when institutional claims are actively misleading or dangerous

• For physician-level procedure coding in the outpatient setting: The institutional claim's
• For comorbidity scoring without POA filtering: Applying the Charlson or Elixhauser index to
• For episode construction without interim-bill deduplication: Using raw institutional claims
• For cost analysis using FL47 charges: Billed charges are not a valid cost measure. Using
• When the payer is Medicare Advantage: MA plans submit encounter data, not FFS claims; the
• For pre-October 2007 POA-based outcome algorithms in Medicare FFS: POA reporting was not

Data-source operational depth

• Medicare FFS MedPAR: The Medicare Provider Analysis and Review file is the pre-processed
• Medicare Outpatient SAF: Contains outpatient hospital claims (TOB 013x, 073x), hospital-based
• Commercial institutional claims (Optum, MarketScan, etc.): Structure mirrors the 837I but
• Medicaid institutional claims: Highly variable by state. TOB coding, POA reporting, and

Licensing note

: The complete UB-04 form and its full code sets are maintained by the National Uniform Billing Committee (NUBC) and are copyrighted by the American Hospital Association. This entry describes field semantics as publicly documented in CMS Medicare Claims Processing Manual (Pub 100-04, Chapter 25) and CMS program memoranda. The AHA/NUBC UB-04 Data Specifications Manual is a licensed product required for implementation-level reference.

Worked example

Scenario

A researcher at a health plan wants to study 30-day all-cause readmissions for Medicare fee-for-service patients hospitalized with acute myocardial infarction (AMI). She pulls the Medicare MedPAR file and finds five claims for patient 0042 across a single calendar month. She needs to determine which of these represent distinct hospitalizations, which should be merged or discarded, and what the correct episode characteristics are for this patient. She then needs to classify the secondary diagnoses correctly for risk adjustment.

Dataset

Five raw UB-04-derived institutional claim records for patient 0042 from the MedPAR/outpatient file (before deduplication). Claim IDs beginning with R indicate replacement of claim A.

Steps

• Claim A001 has TOB 0112 (hospital inpatient, frequency digit 2 = interim-first). This is a partial bill for the beginning of the stay, not a complete admission. Do NOT count it as a separate hospitalization.

• Claim A001-R1 has TOB 0117 (hospital inpatient, frequency digit 7 = replacement). It supersedes A001, covers the full stay from 2023-03-01 through 2023-04-14, and has frequency digit 7 indicating it is the corrected/final version. Keep A001-R1, discard A001. The from-through span is 45 days (2023-03-01 to 2023-04-14), confirming this was a long stay that triggered interim billing. Discharge status 01 = discharged home. This is Hospitalization 1 (AMI index admission).

• Claim A002 has TOB 0131 (hospital outpatient, frequency 1). This is an outpatient visit (from/through = same day), not an inpatient stay. It falls within the post-discharge window and is not a readmission. Exclude from the inpatient readmission denominator; include in outpatient utilization if needed.

• Claim B001 has TOB 0111 (hospital inpatient, admit-through-discharge, frequency 1). From 2023-04-20 to 2023-04-26 = 6 days. Discharge status 02 = transferred to another acute hospital. The secondary diagnosis E11.9 (type 2 diabetes) is coded POA = N, meaning the patient was admitted without diabetes as a coded comorbidity and it emerged during the stay — either a new finding or a documentation gap. Secondary diagnosis E11.9 should be EXCLUDED from the Elixhauser comorbidity score for this admission. This is Hospitalization 2, a potential index readmission (within 30 days of H1 discharge on 2023-04-14).

• Claim C001 has TOB 0111 (hospital inpatient, admit-through-discharge). From 2023-04-26 to 2023-05-03. Admission date (2023-04-26) matches the transfer-out date from B001. Because B001 discharge status = 02 (transfer to another acute hospital) and C001 admission date = B001 through date, B001 and C001 represent a SINGLE EPISODE spanning a hospital-to-hospital transfer. Merge B001 + C001 into one episode: from 2023-04-20, through 2023-05-03, 13 total days; final discharge status = 01 (home) from C001.

• Final episode summary: Hospitalization 1 = A001-R1 (AMI, 2023-03-01 to 2023-04-14, discharged home). Hospitalization 2 = merged B001+C001 (heart failure with transfer, 2023-04-20 to 2023-05-03). Days between H1 discharge (2023-04-14) and H2 admission (2023-04-20) = 6 days. This is a readmission within 30 days = 2023-04-14 + 30 days = 2023-05-14, so H2 qualifies. 6 / 30 = 0.20 of the 30-day window has elapsed at the time of readmission.

• POA-adjusted Elixhauser comorbidity for H2: E11.9 (diabetes, POA = N on B001) is excluded as a hospital-acquired finding; I50.9 (heart failure, POA = Y on C001) is included as a preexisting condition. The comorbidity count changes by 1 depending on whether POA filtering is applied — a difference that can meaningfully shift predicted readmission probability in a risk-adjustment model.

Result

After deduplication and transfer-chain merging: patient 0042 had 2 distinct inpatient episodes. Episode 1 (AMI index): 2023-03-01 to 2023-04-14, 45 days, discharged home. Episode 2 (readmission): 2023-04-20 to 2023-05-03, 13 days, merged from transfer. Days to readmission = 6 days. 6 / 30 = 0.20 fraction of the 30-day window elapsed. POA-adjusted Elixhauser score for episode 2 includes heart failure (POA = Y) but excludes diabetes (POA = N on admission claim). Raw claim count before deduplication was 5; correct episode count is 2.

Timeline Spec

Title

UB-04 institutional claims for patient 0042: deduplication, transfer merge, and readmission

Window

Start

2023-03-01

End

2023-05-14

Label

Index admission through 30-day readmission window

Events

• Label

  A001 (interim, DISCARD)

  Start

  2023-03-01

  Length Days

  31

  Quantity

  TOB 0112 frequency 2 = interim

• Label

  A001-R1 (replacement, KEEP): H1 AMI

  Start

  2023-03-01

  Length Days

  45

  Quantity

  TOB 0117 freq 7 replacement; 45 days

• Label

  A002 (outpatient, not a readmission)

  Start

  2023-04-10

  Length Days

  1

  Quantity

  TOB 0131 outpatient

• Label

  B001: H2 acute HF (transfer out)

  Start

  2023-04-20

  Length Days

  6

  Quantity

  TOB 0111 disch status 02

• Label

  C001: H2 continued (post-transfer)

  Start

  2023-04-26

  Length Days

  7

  Quantity

  TOB 0111 merged with B001

Spans

• Kind

  covered

  Start

  2023-03-01

  End

  2023-04-13

  Label

  H1 AMI inpatient stay (45 days)

• Kind

  gap

  Start

  2023-04-14

  End

  2023-04-19

  Label

  Post-discharge gap (6 days)

• Kind

  covered

  Start

  2023-04-20

  End

  2023-05-02

  Label

  H2 merged transfer episode (13 days)

• Kind

  followup

  Start

  2023-04-14

  End

  2023-05-14

  Label

  30-day readmission window

Result

Label

Readmission at day 6; 6/30 = 0.20 of window elapsed

Value

0.2

Runnable example

import pandas as pd

# ------------------------------------------------------------------
# 1. DEDUPLICATION: resolve interim bills and replacement claims
#    Input: raw institutional claims DataFrame with TOB as a string
#    Key columns: claim_id, patient_id, from_date, through_date, tob,
#                 disch_status, principal_dx, total_charges
# ------------------------------------------------------------------

def parse_tob(tob: str) -> dict:
    """Extract facility type, classification, and frequency from a 4-char TOB string."""
    tob = str(tob).zfill(4)
    return {
        "facility_type": tob[1],       # position 2 (0-indexed 1)
        "classification": tob[2],      # position 3: 1=IP partA, 3=OP, etc.
        "frequency": tob[3],           # position 4: 1=final, 2=first interim, 7=replacement, 8=void
    }

def deduplicate_institutional_claims(df: pd.DataFrame) -> pd.DataFrame:
    """
    Remove interim bills, apply replacement logic, and exclude void claims.
    Returns the deduplicated DataFrame with one row per final admitted episode.

    TOB frequency digit rules (per CMS Claims Processing Manual Ch. 25):
      1 = admit-through-discharge (KEEP as final)
      2 = interim first (DROP — subsumed by frequency-1 final bill)
      3 = interim continuing (DROP)
      4 = interim last (DROP — superseded by frequency-1)
      7 = replacement of prior claim (KEEP as replacement, discard original)
      8 = void/cancel (DROP claim and its predecessor)
    """
    df = df.copy()
    df["tob_freq"] = df["tob"].astype(str).str.zfill(4).str[3]

    # Step 1: drop void claims and the claims they void
    # (in practice, voids reference original claim_id; here we drop by frequency = 8)
    void_ids = set(df.loc[df["tob_freq"] == "8", "claim_id"])
    df = df[~df["tob_freq"].isin(["8"])]
    # If your data has an original_claim_id reference, also drop those originals here.

    # Step 2: for replacement claims (frequency 7), keep only the replacement
    # and discard any earlier versions with the same original_claim_control_number.
    # Here we use claim_id prefix as a proxy (real data uses CLM_ID or ICN).
    replacements = df[df["tob_freq"] == "7"]["claim_id"].str.replace(r"-R\d+$", "", regex=True)
    original_ids_to_drop = set(replacements)
    df = df[~((df["claim_id"].isin(original_ids_to_drop)) & (df["tob_freq"] != "7"))]

    # Step 3: drop interim bills (frequency 2, 3, 4)
    df = df[~df["tob_freq"].isin(["2", "3", "4"])]

    df = df.drop(columns=["tob_freq"])
    return df.reset_index(drop=True)


# ------------------------------------------------------------------
# 2. TRANSFER CHAIN MERGING: fuse status-02 discharge to next admission
#    Input: deduplicated inpatient claims, sorted by patient + from_date
# ------------------------------------------------------------------

def merge_transfer_chains(df: pd.DataFrame, tolerance_days: int = 1) -> pd.DataFrame:
    """
    Merge inpatient claims connected by discharge_status = '02' (transferred to another
    acute hospital) when the receiving admission date is within tolerance_days of the
    transfer-out through_date.

    Returns a DataFrame where transfer chains appear as a single episode with:
      - from_date = first admission date in the chain
      - through_date = last discharge through_date in the chain
      - disch_status = final hospital's discharge status
      - transfer_chain = True if merged from multiple claims
    """
    df = df.copy()
    df["from_date"] = pd.to_datetime(df["from_date"])
    df["through_date"] = pd.to_datetime(df["through_date"])
    df = df.sort_values(["patient_id", "from_date"]).reset_index(drop=True)
    df["transfer_chain"] = False
    df["chain_id"] = range(len(df))

    # Walk patient-by-patient
    merged_rows = []
    for pid, group in df.groupby("patient_id"):
        group = group.reset_index(drop=True)
        i = 0
        while i < len(group):
            row = group.iloc[i].copy()
            # Check if this claim is a transfer-out
            while (str(row["disch_status"]) == "02") and (i + 1 < len(group)):
                next_row = group.iloc[i + 1]
                gap = (next_row["from_date"] - row["through_date"]).days
                if gap <= tolerance_days:
                    # Merge: extend the through_date and take the next claim's discharge status
                    row["through_date"] = next_row["through_date"]
                    row["disch_status"] = next_row["disch_status"]
                    row["transfer_chain"] = True
                    i += 1
                else:
                    break  # Gap too large — not a transfer
            merged_rows.append(row)
            i += 1

    result = pd.DataFrame(merged_rows).reset_index(drop=True)
    return result


# ------------------------------------------------------------------
# 3. POA-ADJUSTED COMORBIDITY FILTERING
#    Apply Elixhauser or Charlson only to POA = Y and exempt diagnoses
# ------------------------------------------------------------------

def filter_poa_comorbidity_dx(
    dx_poa_pairs: list[tuple[str, str]],
    include_unknown: bool = True,
) -> list[str]:
    """
    Given a list of (icd10_code, poa_indicator) pairs from UB-04 secondary dx fields,
    return only the diagnosis codes eligible for comorbidity scoring.

    POA values (CMS documentation):
      Y = present on admission -> include
      N = not present on admission (hospital-acquired) -> EXCLUDE
      U = unknown -> include if include_unknown=True (conservative default)
      W = clinically undetermined -> include if include_unknown=True
      1 = exempt from POA reporting -> include (these codes are exempt by CMS definition)

    Returns list of eligible ICD-10-CM codes for comorbidity mapping.
    """
    include_flags = {"Y", "1"}
    if include_unknown:
        include_flags.update({"U", "W"})

    eligible = [
        dx for dx, poa in dx_poa_pairs
        if str(poa).upper() in include_flags
    ]
    return eligible


# ------------------------------------------------------------------
# EXAMPLE USAGE (mirrors the worked example)
# ------------------------------------------------------------------

if __name__ == "__main__":
    raw_claims = pd.DataFrame({
        "claim_id":    ["A001", "A001-R1", "A002", "B001", "C001"],
        "patient_id":  [42, 42, 42, 42, 42],
        "from_date":   ["2023-03-01", "2023-03-01", "2023-04-10", "2023-04-20", "2023-04-26"],
        "through_date":["2023-03-31", "2023-04-14", "2023-04-10", "2023-04-26", "2023-05-03"],
        "tob":         ["0112", "0117", "0131", "0111", "0111"],
        "disch_status":["30", "01", "01", "02", "01"],
        "principal_dx":["I21.0", "I21.0", "Z00.00", "I50.9", "I50.9"],
    })

    # Step 1: deduplicate
    deduped = deduplicate_institutional_claims(raw_claims)
    print("After deduplication:")
    print(deduped[["claim_id", "from_date", "through_date", "tob", "disch_status"]])
    # Expected: A001-R1 (replacement kept), A002 (outpatient), B001, C001
    # A001 (original of replacement) and interim bills are dropped

    # Step 2: filter to inpatient only (TOB facility_type=1, classification=1 = inpatient Part A)
    ip = deduped[deduped["tob"].str.zfill(4).str[1:3].isin(["11", "12"])].copy()

    # Step 3: merge transfers
    ip_merged = merge_transfer_chains(ip)
    print("\nAfter transfer merge:")
    print(ip_merged[["claim_id", "from_date", "through_date", "disch_status", "transfer_chain"]])
    # B001+C001 merge into one episode with through_date 2023-05-03

    # Step 4: POA filtering example
    secondary_dx_h2 = [("E11.9", "N"), ("I50.9", "Y")]
    eligible = filter_poa_comorbidity_dx(secondary_dx_h2, include_unknown=True)
    print("\nPOA-eligible diagnoses for H2:", eligible)
    # Expected: ['I50.9'] only (E11.9 excluded as POA=N)

library(data.table)

# ------------------------------------------------------------------
# 1. DEDUPLICATION: resolve TOB frequency logic on raw institutional claims
# ------------------------------------------------------------------

deduplicate_institutional_claims <- function(dt) {
  # dt: data.table with columns: claim_id, patient_id, from_date, through_date,
  #     tob (character), disch_status, principal_dx
  dt <- copy(dt)
  dt[, tob_freq := substr(formatC(tob, width = 4, flag = "0"), 4, 4)]

  # Drop void claims (frequency = 8)
  dt <- dt[tob_freq != "8"]

  # For replacement claims (frequency = 7), strip the -R suffix to find originals
  # and drop the original version (keeping the replacement)
  replacements <- dt[tob_freq == "7", gsub("-R[0-9]+$", "", claim_id)]
  dt <- dt[!(claim_id %in% replacements & tob_freq != "7")]

  # Drop interim bills (frequency = 2, 3, 4)
  dt <- dt[!tob_freq %in% c("2", "3", "4")]

  dt[, tob_freq := NULL]
  return(dt)
}


# ------------------------------------------------------------------
# 2. TRANSFER CHAIN MERGING
# ------------------------------------------------------------------

merge_transfer_chains <- function(dt, tolerance_days = 1) {
  dt <- copy(dt)
  dt[, from_date := as.Date(from_date)]
  dt[, through_date := as.Date(through_date)]
  setorder(dt, patient_id, from_date)
  dt[, transfer_chain := FALSE]

  result_list <- list()

  for (pid in unique(dt$patient_id)) {
    grp <- dt[patient_id == pid]
    i <- 1
    while (i <= nrow(grp)) {
      row <- as.list(grp[i])
      # Follow transfer chain
      while (as.character(row$disch_status) == "02" && (i + 1) <= nrow(grp)) {
        nxt <- as.list(grp[i + 1])
        gap <- as.integer(nxt$from_date - row$through_date)
        if (gap <= tolerance_days) {
          row$through_date  <- nxt$through_date
          row$disch_status  <- nxt$disch_status
          row$transfer_chain <- TRUE
          i <- i + 1
        } else {
          break
        }
      }
      result_list[[length(result_list) + 1]] <- as.data.table(row)
      i <- i + 1
    }
  }

  rbindlist(result_list, fill = TRUE)
}


# ------------------------------------------------------------------
# 3. POA-ADJUSTED COMORBIDITY FILTERING
# ------------------------------------------------------------------

filter_poa_comorbidity_dx <- function(dx_vec, poa_vec, include_unknown = TRUE) {
  # dx_vec: character vector of ICD-10-CM codes
  # poa_vec: character vector of POA indicators (Y/N/U/W/1)
  # Returns the subset of dx_vec eligible for comorbidity scoring

  include_flags <- c("Y", "1")
  if (include_unknown) include_flags <- c(include_flags, "U", "W")

  dx_vec[toupper(poa_vec) %in% include_flags]
}


# ------------------------------------------------------------------
# EXAMPLE USAGE
# ------------------------------------------------------------------

raw_claims <- data.table(
  claim_id    = c("A001", "A001-R1", "A002", "B001", "C001"),
  patient_id  = rep(42, 5),
  from_date   = c("2023-03-01", "2023-03-01", "2023-04-10", "2023-04-20", "2023-04-26"),
  through_date = c("2023-03-31", "2023-04-14", "2023-04-10", "2023-04-26", "2023-05-03"),
  tob         = c("0112", "0117", "0131", "0111", "0111"),
  disch_status = c("30",  "01",   "01",   "02",   "01"),
  principal_dx = c("I21.0", "I21.0", "Z00.00", "I50.9", "I50.9")
)

# Step 1: deduplicate
deduped <- deduplicate_institutional_claims(raw_claims)
cat("After deduplication:\n")
print(deduped[, .(claim_id, from_date, through_date, tob, disch_status)])

# Step 2: inpatient only (TOB positions 2-3 = "11")
ip <- deduped[substr(formatC(tob, width = 4, flag = "0"), 2, 3) %in% c("11", "12")]

# Step 3: merge transfers
ip_merged <- merge_transfer_chains(ip)
cat("\nAfter transfer merge:\n")
print(ip_merged[, .(claim_id, from_date, through_date, disch_status, transfer_chain)])

# Step 4: POA filtering
secondary_h2_dx  <- c("E11.9", "I50.9")
secondary_h2_poa <- c("N", "Y")
eligible <- filter_poa_comorbidity_dx(secondary_h2_dx, secondary_h2_poa)
cat("\nPOA-eligible diagnoses for H2:", eligible, "\n")
# Expected: "I50.9" only