Baseline Characteristics and Covariate Balance

In plain language

When a study isn't randomized, the people who got the new drug and the people who got the comparator usually differ before treatment even starts (one group is older, sicker, etc.), and those differences can fake or hide a treatment effect. The baseline ('Table 1') lays out each pre-treatment characteristic side by side for both groups and asks one question: are the groups similar enough to compare fairly? The standard yardstick is the standardized mean difference (SMD), a same-scale gap measure where roughly below 0.10 counts as 'close enough.' One honest caveat: a balanced table only proves the groups match on the things you measured, never on the things you didn't.

In a randomized trial the baseline table documents a known truth (randomization should have balanced everything, measured and unmeasured). In nonrandomized real-world evidence (RWE) it documents the central problem the study exists to solve: treatment was chosen, so the arms differ on the very prognostic factors that drive the outcome. The baseline table and its balance diagnostics are therefore a design instrument, not clerical output — they show who got which treatment, who was excluded by the eligibility cascade, and whether the adjustment strategy (matching, weighting, restriction) made the comparison groups exchangeable on measured covariates within the population the estimand targets.

Core conceptual distinction: balance is a property of an estimator + target population, not of a dataset. Standardized differences must be computed in the same pseudo-population the effect estimate describes. Inverse-probability-of-treatment weighting (IPTW) targets the ATE and balance is assessed in the full weighted sample; standardized-mortality-ratio (SMR/ATT) weights and 1:1 matching target the ATT and balance is assessed in the treated-anchored sample; overlap (ATO) weights target the population with clinical equipoise and balance is assessed in the overlap-weighted sample where, by construction, weighted means are exactly equal for any covariate in the propensity-score (PS) model. Reporting an unweighted Table 1 next to a weighted effect estimate, or pooling matched pairs as if they were the source population, is a category error: the diagnostic no longer corresponds to the estimand. A "balanced" table also says nothing about unmeasured confounding — it is necessary, never sufficient, for valid causal inference.

The right diagnostics (and the cardinal sin)

- Standardized mean difference (SMD): for a continuous covariate, (mean_treated − mean_comparator) / pooled SD; for a binary covariate, (p1 − p0) / sqrt[(p1(1−p1) + p0(1−p0))/2]. It is sample-size-independent, which is exactly why it replaces hypothesis tests. The convention |SMD| < 0.10 flags adequate balance, but it is a heuristic, not a guarantee — a strong confounder at SMD = 0.08 can matter more than a non-confounder at 0.15. - Variance ratio (treated/comparator) close to 1, and higher-moment / distributional checks (eCDF, KS-type distance, quantile-quantile or mirrored histograms): two groups can have identical means yet different spreads or skew, which the SMD of the mean alone hides. Report these for key continuous covariates. - Propensity-score overlap and effective sample size (ESS). Inspect the PS distribution by arm for non-overlap (positivity); after weighting report ESS = (Σw)² / Σw², because a handful of extreme weights can collapse the real information content even when SMDs look pristine. - The cardinal sin: p-values as balance diagnostics. In a 200,000-patient claims cohort a 0.3-year age difference is "p < 0.001" and a 6-year difference can be "p = 0.2" in a 40-patient subgroup. The test answers sample size, not practical comparability. Austin (2009) and consensus guidance reject significance testing of baseline balance; use standardized differences.

Pros, cons, and trade-offs (vs the alternatives you would otherwise reach for)

- vs an omnibus c-statistic / global balance metric (e.g., the PS model AUC or a single Mahalanobis distance): the covariate-by-covariate SMD/variance-ratio table is interpretable and actionable — it names the residually imbalanced confounder so you can enrich the PS model, restrict, or re-specify the comparator. A single global number hides which covariate failed and rewards overfitting the PS. Prefer the per-covariate table for reporting and decisions; use a global summary (max/median SMD, a Love plot) only to summarize a high-dimensional table, never to replace it. - vs a formal balance hypothesis test (Hotelling's T², per-covariate t/chi-square): standardized differences are sample-size-independent and decision-relevant; tests conflate imbalance with power and are explicitly discouraged. There is essentially no setting in large RWE where a balance p-value is the right tool. - vs trusting the outcome model's covariate adjustment to "fix" residual imbalance: regression adjustment on top of a poorly balanced design extrapolates across regions of non-overlap and is fragile to model misspecification. The balance table tells you whether design (matching/weighting/restriction) carried the load so the outcome model only has to interpolate. Prefer design over reliance on modeling when overlap is poor. - Trade-off in what balance buys you: chasing SMD < 0.10 on every one of hundreds of high-dimensional covariates via aggressive trimming or tight calipers shrinks the cohort and ESS, widens confidence intervals, and shifts the estimand toward an ever-narrower overlap population (Stürmer 2010). Balance is purchased with generalizability and precision; the table should be read alongside ESS and the change in cohort composition, not optimized in isolation.

When to use

Always, as a mandatory deliverable, in any nonrandomized comparative-effectiveness or safety study and in any descriptive cohort comparison: (1) an unadjusted/design table to expose treatment channeling and the eligibility cascade, and (2) an adjusted table (matched/weighted) computed in the estimand's population to certify that the balancing step worked before any outcome model is fit. It is the standard acceptance gate for PS matching/IPTW/overlap weighting and for target-trial emulations.

When NOT to use / when it is actively misleading or dangerous

- Do not treat a balanced table as evidence of no confounding. Measured balance is silent on unmeasured confounders (frailty, performance status, over-the-counter use, indication severity not coded in claims). Pairing a pristine Love plot with a causal claim, absent negative controls / quantitative bias analysis, is the most common over-reach reviewers punish. - Do not assess balance in the wrong population. An unweighted Table 1 reported beside an ATE/IPTW estimate, or a matched-pair table read as if it described the source cohort, misrepresents who the effect is about. - Do not check balance only on the covariates already in the PS model. Overlap weights and a saturated PS force those to balance by construction; the informative check is on prognostic variables excluded from the model and on transformations/interactions, where residual imbalance actually lives. - Do not over-trim to manufacture balance when it shrinks ESS and changes the target population — you may "win" the table and lose the question (Stürmer 2010). - Beware balance achieved on a mis-defined baseline window. Covariates measured after time zero (e.g., a lab drawn the week after initiation) introduce immortal-time and mediator adjustment; a beautiful balanced table built on post-baseline data is balanced on the wrong thing.

Data-source operational depth — real failure modes and workarounds

- Claims (FFS): the strongest measured confounders are usually utilization (prior hospitalizations, ED visits, outpatient counts), cost, medication classes, and diagnosis/procedure counts in a fixed lookback — not raw comorbidity flags, which are noisy. Include these in Table 1 and the PS. Failure mode: Medicare Advantage (MA) person-time lacks fee-for-service (FFS) claims, so MA enrollees have artifactually "clean" baselines (few coded comorbidities), creating spurious balance and differential measurement error if MA mix differs by arm. Workaround: restrict to enrollees with complete FFS Parts A/B/D (or a commercial benefit) across the full lookback, and report the MA share by arm. Failure mode: a fixed lookback with variable enrollment makes counts depend on observed time, not true burden; workaround: require continuous enrollment across the lookback or model person-time, and never let enrollment length differ systematically by arm. - EHR: labs, vitals, stage, ECOG, smoking, and PROs are powerful confounders but missing not at random — a missing lab often marks a healthier or less-engaged patient, and missingness can itself differ by arm and by site. Failure mode: complete-case Table 1 silently restricts to the sickest, most-worked-up patients. Workaround: report a missingness indicator per covariate as its own row in Table 1, balance the missingness indicators, and use multiple imputation (or the missing-indicator method only when missingness is plausibly a measured marker), with sensitivity analyses. Visit-driven capture also means baseline "absence of disease" can be absence of contact. - Registry: rich clinical staging/biomarkers (often the true confounders) but typically thin on full pharmacy/utilization; workaround: link to claims for baseline HCRU and to a death index. Failure mode: voluntary-enrollment registries select on prognosis, so balance within the registry may not transport. - Linked claims–EHR–registry: the ideal substrate (severity + completeness + mortality), but linkage selects the linkable subset and date discrepancies (order vs fill vs service) can push covariates across the time-zero boundary; reconcile dates and report balance both in the linked subset and against the unlinked source to gauge selection.

Worked claims example (end to end)

Comparative safety of SGLT2 inhibitors vs DPP-4 inhibitors on lower-limb amputation among adults with type 2 diabetes in a commercial + Medicare FFS database. (1) Cohort: active-comparator, new-user design — first fill (`fill_date`) of either class as `index_date`; require 365 days of continuous medical + pharmacy enrollment with no MA-only person-time and no prior fill of either class in the lookback (true washout). (2) Baseline window: covariates measured only in [index_date − 365, index_date] — age, sex, region, prior amputation/PAD diagnoses, neuropathy, insulin/metformin use, HbA1c proxy, counts of inpatient/ED/outpatient visits, total paid cost, and a missing-HbA1c indicator. (3) Unadjusted Table 1: SGLT2 initiators are younger, lower prior-amputation rate, fewer hospitalizations — classic channeling (age SMD ≈ 0.42, prior-PAD SMD ≈ 0.28). (4) Adjustment: fit a high-dimensional PS, apply overlap (ATO) weights for an equipoise estimand. (5) Adjusted Table 1: weighted SMDs for all PS covariates ≈ 0.00 by construction; the informative check is on excluded prognostic proxies (e.g., wound-care procedure codes) and variance ratios — all < 0.10 and ~1.0 here. (6) Guardrails: report ESS (e.g., 18,400 → 12,950 weighted) and the change in weighted-mean age vs the source to show how ATO reshaped the population; only then fit the outcome model. A worked SMD: in the lookback, mean total cost \$14,200 (treated) vs \$11,800 (comparator), pooled SD \$9,000 → SMD = 0.27 (clear channeling); after ATO weighting \$12,400 vs \$12,250, pooled SD \$9,000 → SMD = 0.017 (practically balanced), while the same contrast may still be "p < 0.001" in 30,000 patients — which is precisely why the SMD, not the p-value, governs the decision.

Worked example

Scenario

We are comparing a new diabetes drug (treated arm, 500 patients) against an older comparator (500 patients) in a claims database. Before we trust any outcome comparison, we build Table 1 on three baseline characteristics measured before each patient's first fill: age, percent female, and percent with a pre-existing diabetes complication. For each one we compute an SMD to see whether the groups are comparable, then repeat after applying weights designed to make the groups match.

Dataset

A baseline ('Table 1') summary an analyst would actually report: group means/percentages plus the SMD column, shown before and after weighting.

Steps

• The SMD for a continuous covariate is (mean in treated minus mean in comparator) divided by the pooled standard deviation, the combined typical spread of the two groups.

• Work the age row before weighting: the treated group averages 61 years, the comparator 67. The two group spreads (SD 11 and 12) combine into a pooled SD of sqrt((11^2 + 12^2)/2) = sqrt(132.5) = 11.51 years.

• So the age SMD is (61 - 67) / 11.51 = -6 / 11.51 = -0.52. The sign just says treated is younger; we read the size, 0.52.

• Apply the balance rule: an absolute SMD below 0.10 is the convention for 'close enough.' Before weighting, age (0.52) and the diabetes complication (0.31) both blow past 0.10, so those groups are NOT comparable, while percent female (0.08) is already under the line.

• After weighting, recompute on the rebalanced groups. Age becomes (64 - 64.5) / 11.51 = -0.04, and the diabetes complication shrinks to 0.04. All three covariates are now below 0.10.

Result

Before weighting, the groups were imbalanced on age (|SMD| = 0.52) and diabetes complication (|SMD| = 0.31) and balanced on sex (|SMD| = 0.08) - a clear sign of treatment channeling toward younger, healthier patients. After weighting, all three covariates fall below the 0.10 cut-off (age -0.04, female -0.04, diabetes -0.04), so the table now passes the measured-balance check. Important caveat: this only certifies the three covariates we measured; it says nothing about unmeasured differences like frailty.

Runnable example

import numpy as np
import pandas as pd


def _wmean_wvar(x, w):
    m = np.average(x, weights=w)
    v = np.average((x - m) ** 2, weights=w)  # weighted (biased) variance; adequate for SMD
    return m, v


def standardized_diff(x, treated, weight=None, binary=False):
    x = np.asarray(x, dtype=float)
    a = np.asarray(treated)
    w = np.ones(len(x)) if weight is None else np.asarray(weight, dtype=float)
    m1, v1 = _wmean_wvar(x[a == 1], w[a == 1])
    m0, v0 = _wmean_wvar(x[a == 0], w[a == 0])
    if binary:
        # proportions: pooled SD from p(1-p), not the raw variance
        denom = np.sqrt((m1 * (1 - m1) + m0 * (1 - m0)) / 2)
    else:
        denom = np.sqrt((v1 + v0) / 2)
    smd = np.nan if denom == 0 else (m1 - m0) / denom
    vratio = np.nan if (binary or v0 == 0) else v1 / v0  # variance ratio only meaningful for continuous
    return smd, vratio


def effective_sample_size(weight):
    w = np.asarray(weight, dtype=float)
    return (w.sum() ** 2) / np.sum(w ** 2)  # ESS; falls sharply with extreme weights


def balance_table(df, treated_col, covariates, binary_cols=(), weight_col=None):
    w = df[weight_col].to_numpy() if weight_col else None
    rows = []
    for c in covariates:
        smd, vr = standardized_diff(df[c], df[treated_col], w, binary=c in set(binary_cols))
        rows.append({"covariate": c, "abs_smd": abs(smd), "variance_ratio": vr,
                     "balanced": abs(smd) < 0.10})
    out = pd.DataFrame(rows).sort_values("abs_smd", ascending=False)
    if weight_col:  # report ESS by arm so a balanced table is not read in isolation
        for arm in (1, 0):
            ess = effective_sample_size(df.loc[df[treated_col] == arm, weight_col])
            out.attrs[f"ess_arm_{arm}"] = ess
    return out


# balance_table(cohort, "treated",
#               covariates=["age", "lookback_cost", "n_inpatient", "prior_pad", "missing_hba1c"],
#               binary_cols=["prior_pad", "missing_hba1c"],
#               weight_col="overlap_weight")

library(WeightIt)
library(cobalt)
library(tableone)

covs <- c("age", "sex", "lookback_cost", "n_inpatient", "prior_pad", "missing_hba1c")

# Estimate overlap (ATO) weights from a high-dimensional-style PS model.
w.out <- weightit(treated ~ age + sex + lookback_cost + n_inpatient + prior_pad + missing_hba1c,
                  data = df, method = "glm", estimand = "ATO")

# Balance table: weighted + unweighted SMDs, variance ratios, KS distance, and effective sample size.
bal <- bal.tab(w.out, stats = c("mean.diffs", "variance.ratios", "ks.statistics"),
               un = TRUE, disp = c("means"), thresholds = c(m = 0.10, v = 2))
print(bal)                       # includes Adjusted/Unadjusted ESS rows
love.plot(w.out, stats = "mean.diffs", thresholds = c(m = 0.10),
          abs = TRUE, var.order = "unadjusted")

# Companion stratified Table 1 with SMDs (no significance tests in large RWE).
print(CreateTableOne(vars = covs, strata = "treated", data = df, test = FALSE), smd = TRUE)