Marginal Effects and Interpretation of Inferential Statistics

In plain language

When a study compares two treatments, the statistical model produces an odds ratio (OR) — a ratio that lives on a math scale, not the familiar percentage-point scale that clinicians and payers use to make decisions. An average marginal effect (AME) translates that OR into an absolute risk difference: how many more (or fewer) patients out of 100 would experience the outcome if everyone received Drug A instead of Drug B. The translation matters because an OR of 2.0 can mean a 8-percentage-point increase in risk for low-risk patients but a 17-percentage-point increase for high-risk patients — the OR hides that difference, while the AME surfaces it by averaging across the real mix of patients in the study.

Almost every RWE analysis ends in a regression model — logistic for binary safety/effectiveness outcomes, Poisson or negative binomial for healthcare-resource-utilization (HCRU) counts, Cox for time-to-event, linear for continuous scores. The number that falls out of `summary(model)` is a conditional coefficient: the effect holding the model's other covariates fixed, expressed on the model's own (usually non-linear) link scale as an odds ratio, hazard ratio, or incidence-rate ratio. Decisions — formulary, label, guideline, value dossier — are made at the population level, on an absolute scale, for a defined target group. The bridge between the two is the marginal (population-averaged) effect, obtained by standardization (g-computation): predict each person's outcome under treatment and under control using their own observed covariates, then average the contrast across the population. This is the disciplined way to turn an OR into a number a payer can act on.

Core estimand distinction

A conditional effect is defined within strata of the adjustment covariates (the log-OR for a 70-year-old man with two comorbidities). A marginal effect is the average over a specified covariate distribution: the Average Marginal Effect (AME) averages the individual-level contrasts over the observed population (predict under A=1 for everyone, predict under A=0 for everyone, average the difference), while the Marginal Effect at the Mean (MEM) plugs population means into a single prediction. For linear models with no interactions the two coincide and equal the coefficient. For non-linear models they diverge because of non-collapsibility: the conditional OR and HR are not weighted averages of stratum-specific effects even with no confounding, so a "fully adjusted" OR of 0.70 is not the marginal OR, and certainly is not a 30% absolute risk reduction. AME is generally preferred over MEM because the "mean" patient (e.g., 0.4 of a sex, 1.7 comorbidities) corresponds to no real person and behaves badly under interactions. The scale matters too: standardize to a risk difference / risk ratio (logistic), an expected-count or rate difference (Poisson/NB), or a survival-probability difference or restricted-mean-survival-time (RMST) difference at a fixed horizon (Cox) — the last two are marginal summaries that remain interpretable when proportional hazards fails, which the HR does not. Crucially, marginal effects are a reporting/standardization step; they inherit causality only from the design (no unmeasured confounding, positivity, correct intercurrent-event handling). Standardizing a garbage model produces a precise, collapsible, garbage answer.

Pros, cons, and trade-offs

- vs reporting raw model coefficients (OR / HR / IRR): Marginal effects are collapsible and aggregatable, sit on the scale clinicians and payers reason with, and dissolve the "odds-ratio looks like a relative risk" trap. Cost: more computation, you must name the population averaged over, and variance needs the delta method or bootstrap rather than the model's printed SE. Prefer marginal effects as the headline for any decision-facing RWE; keep coefficients for model transparency, not as the answer. - vs marginal effects at the mean (MEM) / "typical patient" values: AME respects the real covariate distribution and is correct under interactions and non-linearities; MEM is one cheap prediction at a possibly fictional patient. Prefer AME for inference; reserve MEM/representative-value effects for quick communication, always paired with the AME. - vs conditional / subgroup-specific effects: A single marginal number answers the population question directly but can average away genuine effect modification (by age, line of therapy, biomarker). Report both the overall marginal effect and pre-specified subgroup marginal effects; do not let one collapsed number hide qualitative interaction. - vs IPTW/standardized contrasts from a propensity model: Outcome-regression standardization (g-computation) and IPTW target the same marginal estimand by different nuisance models; combining them (doubly robust / AIPW / TMLE) is consistent if either model is right. Prefer the doubly robust marginal effect when feasible, falling back to plain g-computation only when the outcome model is well understood.

When to use

Any RWE study whose deliverable is a decision — comparative safety/effectiveness, HCRU or cost-offset estimates, label or HTA evidence — should report the marginal effect on an absolute scale, with an explicit statement of the population standardized to. Use it whenever a non-linear model (logistic, Cox, Poisson with covariates) is fit and the audience needs absolute risk, number-needed-to-treat, events avoided, or months gained. Use survival-probability or RMST differences whenever proportional hazards is doubtful or the time horizon is policy-relevant.

When NOT to use — and when it is actively misleading or dangerous

Do not present a marginal effect as causal when the design cannot support it: standardization over a multivariable model fit to a confounded cohort yields a precise associational number that looks like an ATE and will be read as one — this is the most dangerous failure mode here. Do not standardize over a population that includes regions of non-positivity (treatment levels never observed for some covariate patterns); the prediction extrapolates off-support and the "marginal effect" is model fiction. Do not report a single marginal effect in the presence of strong, pre-specified effect modification — it can sit at the null while both subgroups show large opposite effects. Do not standardize over the wrong population: an AME over the full cohort answers an ATE question, not the ATT a payer asked about (then standardize over the treated only). And do not use the model's printed coefficient SE for the marginal effect's confidence interval — it ignores the averaging step and, if a propensity score or weights were estimated, their uncertainty too.

Data-source operational depth

- Claims (FFS vs MA vs commercial): The standardization population must be the actual analytic cohort — continuously enrolled new users, not "everyone with a claim." Medicare Advantage enrollees lack complete FFS person-time, so including MA-only time silently changes the population you are averaging over and biases both the outcome model and the marginal estimate; restrict to fully observable enrollment or report the population explicitly. For Cox-based survival standardization, differential competing risks by exposure (e.g., one drug used in frailer, higher-mortality patients) mean the cause-specific marginal survival difference and the cumulative-incidence (subdistribution) marginal difference answer different questions — pick the estimand before standardizing. Days-supply artifacts (90-day mail order, sample fills) distort the at-risk denominator feeding a Poisson/NB rate standardization. - EHR: Richer covariates sharpen the conditional model and thus the predictions, but irregular, visit-driven capture means the prediction step must handle missingness deliberately (multiple imputation, or last-value-with-indicator) — a naive complete-case standardization averages over the observed-data population, not the target population. Loss to follow-up is potentially informative and must be addressed (IPCW) before a survival-scale marginal effect is credible. - Registry: Often the cleanest population for the target of standardization (adjudicated outcomes, complete severity), so registry covariate distributions are frequently used to transport a marginal effect estimated elsewhere; weak pharmacy capture limits exposure-scale work without claims linkage. - Linked claims–EHR–registry: Best substrate (severity + completeness + mortality), but linkage selection means the standardization population is the linkable subset — state this, since the marginal effect is only marginal over those people.

Worked claims example

Question: 1-year risk of hospitalized heart failure, second-generation sulfonylurea vs DPP-4 inhibitor, among commercial + Medicare-FFS adults with type 2 diabetes. (1) Cohort: ≥18 years, ≥2 diabetes diagnoses, and 365 days of continuous medical+pharmacy (or A/B/D) enrollment before the first qualifying fill; exclude MA-only person-time so "no prior fill" and follow-up are truly observable. (2) Index/time-zero = first fill of either drug; `arm` from the dispensed NDC. (3) Baseline covariates measured only in [index_date−365, index_date] (age, sex, prior insulin, renal dx, prior HF, HCRU intensity), feeding a logistic outcome model for the binary 365-day HF event with censoring handled by requiring 365 days of post-index observable time (or moving to a Cox/RMST standardization if censoring is heavy). (4) Fit the outcome model (optionally on a PS-weighted cohort for double robustness). (5) Standardize: copy the analytic frame twice, set `arm='STUDY'` in one and `arm='COMPARATOR'` in the other, predict 365-day HF risk for every person under both, average each. Marginal risk under study = 0.058, under comparator = 0.041 → marginal risk difference = +0.017 (1.7 excess HF hospitalizations per 100 patients per year; NNH ≈ 59), with a bootstrap 95% CI resampling persons (and refitting the PS if doubly robust). Report this alongside the conditional adjusted OR (e.g., 1.46) and state explicitly: marginal over the treated-eligible new-user cohort, causal only under the design's no-unmeasured-confounding and positivity assumptions.

Interpreting the output

Consider the logistic-model example with OR = 2.0 applied to a mixed-risk cohort: 50 low-risk patients (baseline risk 10%) and 50 high-risk patients (baseline risk 40%). Standardizing predicted probabilities across both copies of the data yields marginal risk under treatment = 0.376 and under control = 0.250, giving an average marginal effect (AME) = +0.126, or 12.6 excess events per 100 patients.

(1) Formal statistical interpretation. The AME of +0.126 is the average, across all patients in the analytic sample, of the individual predicted risk difference attributable to treatment. Unlike the OR of 2.0, the AME is on an absolute risk-difference scale and is not subject to non-collapsibility: it can be directly added and subtracted, varies with covariate distribution, and is a valid average even when the patient-level effects are heterogeneous. Its uncertainty should be quantified with a bootstrap or delta-method confidence interval. Association or causal language depends on the study design and the assumptions invoked.

(2) Practical interpretation for a decision-maker. An OR of 2.0 sounds like "twice the risk," but in a mixed-risk population it translates to about 13 extra events per 100 patients treated — not 100 extra events per 100. For coverage decisions, formulary placement, or number-needed-to-treat calculations, the AME is the right number: it averages over the actual patient mix in the study, making it directly comparable to observed event rates and budget-impact projections.

Worked example

Scenario

A claims study enrolls 100 adults with hypertension. The investigators want to know whether Drug A raises the 1-year risk of a hospital visit compared with Drug B. They fit a logistic regression model adjusting for age and comorbidity burden. The model returns an odds ratio of 2.0 for Drug A vs Drug B. But the cohort has two kinds of patients: 50 are low-risk (10% baseline chance of hospitalization on Drug B) and 50 are high-risk (40% baseline chance). The goal is to show that OR = 2.0 means very different things for each group, and that the AME gives a single honest population summary.

Dataset

Predicted-probability output frame produced after g-computation on 4 representative patients (2 per risk group). Each patient is scored twice: once with arm set to Drug A, once with arm set to Drug B.

Steps

• Convert OR=2.0 to a probability for a low-risk patient (p0=0.10). Odds under Drug B = 0.10 / 0.90 = 0.111. Multiply by OR: 0.111 x 2.0 = 0.222. Convert back to probability: 0.222 / (1 + 0.222) = 2/11 = 0.182. Risk difference for this patient = 0.182 - 0.100 = 0.082.

• Repeat for a high-risk patient (p0=0.40). Odds under Drug B = 0.40 / 0.60 = 0.667. Multiply by OR: 0.667 x 2.0 = 1.333. Convert back: 1.333 / (1 + 1.333) = 4/7 = 0.571. Risk difference for this patient = 0.571 - 0.400 = 0.171.

• The same OR of 2.0 produces an 8.2-percentage-point increase for the low-risk patient but a 17.1-percentage-point increase for the high-risk patient — the OR alone cannot tell you which group you are dealing with.

• Compute the AME by averaging across all 100 patients (50 low-risk, 50 high-risk). Average p1 = (50 x 0.182 + 50 x 0.571) / 100 = (9.10 + 28.55) / 100 = 37.65 / 100 = 0.376. Average p0 = (50 x 0.100 + 50 x 0.400) / 100 = (5.00 + 20.00) / 100 = 25.00 / 100 = 0.250.

• AME (risk difference) = 0.376 - 0.250 = 0.126.

Result

The average marginal effect is a risk difference of +0.126, meaning Drug A is associated with 12.6 additional hospitalizations per 100 patients per year compared with Drug B, averaged over this cohort's actual mix of low- and high-risk patients. This is more policy-interpretable than OR=2.0 because a payer or clinician can immediately ask 'is 12.6 per 100 acceptable?' — they cannot make that judgment from an odds ratio, which shifts between 8.2 and 17.1 percentage points depending on who is in the room.

Runnable example

import numpy as np
import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf

COVARS = ["age", "sex", "prior_insulin", "renal_dx", "prior_hf", "hcru_intensity"]
FORMULA = "event ~ arm + " + " + ".join(COVARS)  # add arm:covar terms if effect modification is expected

def standardized_risk_difference(df: pd.DataFrame) -> dict:
    """Marginal (population-averaged) risk under arm=1 and arm=0 via g-computation."""
    model = smf.glm(FORMULA, data=df, family=sm.families.Binomial()).fit()
    d1, d0 = df.copy(), df.copy()
    d1["arm"], d0["arm"] = 1, 0                      # set everyone to study, then to comparator
    r1 = model.predict(d1).mean()                   # marginal risk if all treated with study
    r0 = model.predict(d0).mean()                   # marginal risk if all treated with comparator
    return {"risk_study": r1, "risk_comp": r0, "risk_difference": r1 - r0,
            "risk_ratio": r1 / r0, "nnh": 1.0 / (r1 - r0) if r1 != r0 else np.inf}

def bootstrap_ci(df: pd.DataFrame, n_boot: int = 1000, seed: int = 1) -> tuple:
    rng = np.random.default_rng(seed)
    ids = df["person_id"].to_numpy()
    diffs = []
    for _ in range(n_boot):
        samp = df.iloc[rng.integers(0, len(df), len(df))]   # resample persons with replacement
        diffs.append(standardized_risk_difference(samp)["risk_difference"])
    return tuple(np.percentile(diffs, [2.5, 97.5]))

point = standardized_risk_difference(df)
lo, hi = bootstrap_ci(df)
print(f"Marginal RD = {point['risk_difference']:.4f} (95% CI {lo:.4f}, {hi:.4f}); NNH = {point['nnh']:.0f}")

library(survival)

COVARS <- c("age", "sex", "prior_insulin", "renal_dx", "prior_hf", "hcru_intensity")
f_logit <- as.formula(paste("event ~ arm +", paste(COVARS, collapse = " + ")))

# --- Marginal risk difference (binary outcome) via g-computation ---
std_rd <- function(df) {
  fit <- glm(f_logit, data = df, family = binomial())
  d1 <- transform(df, arm = factor("STUDY",      levels = levels(df$arm)))
  d0 <- transform(df, arm = factor("COMPARATOR", levels = levels(df$arm)))
  r1 <- mean(predict(fit, d1, type = "response"))   # standardize to study for all
  r0 <- mean(predict(fit, d0, type = "response"))   # standardize to comparator for all
  r1 - r0
}

# --- Marginal RMST difference at a horizon (time-to-event), robust to non-PH ---
std_rmst <- function(df, horizon = 365) {
  fit <- coxph(as.formula(paste("Surv(time, event) ~ arm +",
                                paste(COVARS, collapse = " + "))), data = df)
  rmst_arm <- function(level) {
    nd  <- transform(df, arm = factor(level, levels = levels(df$arm)))
    sf  <- survfit(fit, newdata = nd)                       # one survival curve per person
    smean <- rowMeans(summary(sf, times = horizon)$surv)    # marginal S(horizon)
    # trapezoidal RMST up to horizon from the averaged survival curve:
    tt <- sort(unique(pmin(df$time, horizon)))
    avg_surv <- rowMeans(summary(sf, times = tt)$surv)
    sum(diff(c(0, tt)) * head(c(1, avg_surv), length(tt)))
  }
  rmst_arm("STUDY") - rmst_arm("COMPARATOR")
}

rd   <- std_rd(df)
drm  <- std_rmst(df, horizon = 365)
# Person-level bootstrap CI (resample person_id, refit, re-standardize):
ids  <- unique(df$person_id)
bs   <- replicate(1000, std_rd(df[df$person_id %in% sample(ids, replace = TRUE), ]))
cat(sprintf("Marginal RD = %.4f (95%% CI %.4f, %.4f); RMST diff = %.1f days\n",
            rd, quantile(bs, .025), quantile(bs, .975), drm))