Descriptive Statistics

Descriptive statistics

answer a single question: "what does this variable look like?" They make no causal claim, test no hypothesis, and require no comparison group. Every downstream analytic step — from a Table 1 to a regression to an economic model — rests on a prior choice about how to summarize the raw distribution, and wrong choices introduce errors that compound through the whole analysis.

This entry covers the full toolkit: measures of location, spread, and shape; categorical variable summaries; the Table 1 conventions used in real-world evidence (RWE) and health economics and outcomes research (HEOR); and the most consequential trade-offs, especially the mean-vs-median decision for right-skewed healthcare cost and utilization data.

Core conceptual distinctions

Three questions about any variable answer different questions and require different summaries.

Where is it centered? The arithmetic mean (sum divided by count) is sensitive to every value, including extreme ones. Pull one hospitalization cost to \$500,000 and the mean jumps even if 99 patients had modest bills. The median (middle value after sorting, or the average of the two middle values when n is even) is resistant: it reports what a typical patient experienced. The mode (most frequent value) is rarely used for continuous data but matters for discrete counts and categorical variables.

How spread out is it? The standard deviation (SD) is the square root of the average squared distance from the mean. Because it uses the mean as its anchor, it is equally sensitive to outliers. For a normal distribution, mean ± 1 SD contains approximately 68% of values, mean ± 2 SD contains approximately 95%, and mean ± 3 SD contains approximately 99.7% — the 68-95-99.7 rule. This heuristic fails completely on skewed data: a cost distribution with mean \$12,000 and SD \$28,000 implies a lower bound of mean − 1 SD = −\$16,000, which is meaningless.

When data are skewed, report the interquartile range (IQR): the spread from the 25th percentile (Q1) to the 75th percentile (Q3). The IQR describes the middle 50% of the distribution regardless of how extreme the tails are. The range (minimum to maximum) is sensitive to single extreme values and is usually relegated to a footnote. The coefficient of variation (CV) = SD / mean expresses spread as a fraction of the mean — useful when comparing variability across variables measured on different scales.

What shape does it have? Skewness describes asymmetry: a right-skewed (positive-skew) distribution has a long right tail (mean > median), which is the default for healthcare costs, length of stay, and utilization counts. A left-skewed distribution has a long left tail.

Kurtosis

describes tail weight: high kurtosis ("leptokurtic") means more extreme values than a normal distribution would predict. Concretely, look at a histogram first and a boxplot second — these visual checks tell you whether the normal approximation applies before you decide which summary to use.

The skewed-cost-data rule and the HEOR nuance

The single most important application of this principle in RWE and HEOR is the following: healthcare costs, length of stay, and service-utilization counts are almost always right-skewed because a small fraction of patients generate catastrophically large bills while the majority have modest expenditures. For describing a typical patient's experience, report median and IQR. The mean inflates the central tendency estimate because a handful of \$200,000 admissions pull it far above what most patients incur.

However — and this is a genuinely important nuance for payers and budget impact — the payer pays the mean, not the median. If 1,000 patients are enrolled, the plan's total expenditure is 1,000 × mean, not 1,000 × median. For budget impact analyses, cost-effectiveness models, and any setting that requires total cost estimation, the mean is the appropriate summary because the mean × count = total.

For these analyses, report both the mean and median: the median characterizes the typical patient, the mean drives the budget math. When the two diverge substantially, the distribution is skewed enough that the choice of summary matters and should be explained in the methods.

Categorical variables

For binary or multi-category variables, the right summaries are counts (n) and proportions (%). The single most common error is ambiguous denominators: a "30% rate of prior hospitalization" is meaningless without knowing 30% of what — the full cohort? the treated arm? patients with at least 12 months of lookback? Report the denominator explicitly.

Missingness is its own category

: if 15% of patients lack a recorded HbA1c, that 15% should appear as "missing: n (15%)" in the table, not be silently excluded from the denominator. Collapsing missing patients into a lower denominator inflates the apparent proportion of observed values.

Table 1 conventions in RWE/HEOR

The standard reporting convention for a "Table 1" (baseline characteristics) is:

• Approximately normally distributed continuous variables: mean (SD)
• Skewed continuous variables (costs, LOS, utilization counts): median [IQR] — many journals use square brackets for IQR to distinguish from the round-bracket mean (SD) convention
• Categorical variables: n (%)
• For cost variables specifically: consider reporting both mean (SD) and median [IQR] in the same row

Never report standard error of the mean (SEM) in Table 1

The SEM describes the precision of the estimated mean as an estimator — it shrinks as sample size grows. The SD describes the spread of the actual patient population and does not depend on sample size. Table 1 is supposed to show what the patients look like; the SD does that.

A 50,000-patient claims cohort with SD = \$28,000 and SEM = \$126 should report SD, because SEM = \$126 falsely implies all patients had near-identical costs. This confusion between SEM and SD is one of the most common errors in reporting of clinical and HEOR data.

Descriptive ≠ inferential: no p-values needed, and why that matters

Descriptive statistics describe. No p-value is needed to state that the mean age is 62 years or that 35% of patients had a prior hospitalization. The long-running debate about whether to include p-values in Table 1 of a randomized trial (testing whether randomization succeeded) does not apply to RWE: in a non-randomized study, a significant p-value on a baseline covariate is expected (treatment was chosen, not assigned) and uninformative.

Whether to worry about a 3-year age difference or a 0.5-year difference depends on whether age is a confounder for the outcome of interest — a clinical judgment, not a significance test. In non-randomized RWE, standardized mean differences (SMDs) replace p-values for assessing whether groups are comparable (see baseline-characteristics-and-covariate-balance-rwe).

Pros, cons, and trade-offs — specific and comparative

• Mean vs median: The mean is the natural input to budget math (mean × n = total), and means are required for most parametric statistical tests. The median is robust to outliers and is the appropriate summary of a typical patient's experience when the distribution is skewed. Use the mean when the downstream analysis requires totals or expected values; use the median when you want to characterize what a typical patient experienced. For costs and utilization in RWE/HEOR, routinely report both.
• SD vs IQR: The SD is the natural companion to the mean and is required when computing confidence intervals under normality. The IQR does not assume any distributional shape and is the appropriate spread measure for skewed data. Use SD for normal or near-normal data; use IQR for skewed data and for all cost/LOS/utilization variables in RWE.
• SEM vs SD (the cardinal confusion): The SEM is used in confidence intervals for the mean, not in Table 1. Reporting SEM instead of SD makes patient populations appear artificially homogeneous because SEM shrinks with larger samples. Always use SD in descriptive tables.
• Descriptive vs causal summaries: Descriptive statistics are robust precisely because they make no causal assumptions — but that is also their limitation. A table showing that treated patients had higher average costs does not mean the treatment caused higher costs; confounding by indication is present and untouched. Descriptive summaries are necessary, not sufficient, for causal inference.
• Pooling vs stratifying: Summarizing a bimodal or multimodal distribution with a single mean hides the structure. If men and women have systematically different costs, a pooled mean obscures what is happening in each group. When descriptive summaries are used to characterize a heterogeneous population, stratify on the key variable rather than pool.

When NOT to use — and when descriptive summaries are actively misleading

• Do not use descriptive between-group differences to support causal claims. A mean cost difference of \$5,000 between treated and untreated groups in an observational study reflects confounding by indication, severity of illness, and every other baseline difference — not necessarily the treatment effect. Descriptive statistics set the stage; they do not answer the causal question.
• Do not report means on heavily right-skewed data without also reporting the median. When the mean is substantially higher than the median, readers who see only the mean will overestimate what a typical patient spends. In cost data, report both.
• Do not report means on censored cost data without appropriate adjustment. When follow-up time is censored (as in most survival or time-to-event analyses with administrative censoring), a simple arithmetic mean of observed costs underestimates the true mean cost over the full follow-up period. Inverse probability of censoring weighting (IPCW) or partitioned survival methods are needed for unbiased cost estimation (see healthcare-costs-pppm-pppy-pmpm).
• Do not pool a bimodal or multimodal distribution into one mean. If a variable has two distinct modes (for example, costs from a procedure-naive group and a procedure-heavy group), the mean of the pooled distribution may correspond to no actual patient's experience and will mislead readers who interpret it as a typical value. Stratify or visually inspect the histogram first.
• Do not confuse SEM with SD in Table 1 — this is the most common descriptive statistics error in clinical and HEOR publications, and it systematically understates population heterogeneity.
• Do not describe with ambiguous denominators. Any percentage must specify what the denominator is — total cohort, treated arm, patients with complete data — or the percentage is uninterpretable.

Data-source operational depth

• Claims (FFS): Most continuous variables of interest — costs, length of stay, service counts — are right-skewed in claims data, so median [IQR] is the default. Cost variables require specifying which perspective is being reported (paid amount vs charged amount vs allowed amount) and which services are included (medical only, pharmacy only, or combined). Costs in claims are often expressed per-member-per-month (PMPM) or per-patient-per-year (PPPY) to normalize across enrollment durations.

Descriptive statistics on cost variables should always examine whether a small number of extremely high-cost enrollees are driving the mean (truncation or winsorization may be needed before modeling). Categorical variables such as diagnosis presence/absence are straightforward n (%) but the denominator must reflect the observable enrollment window: a 12-month lookback requires 12 months of continuous enrollment, and restricting to patients with the full lookback affects the n in the denominator. - EHR: Laboratory values, vitals, and clinical scores are often closer to normal than cost data, making mean (SD) more appropriate — but always inspect for skewness and confirm via histogram.

A critical issue with EHR-derived descriptive statistics is missingness: a lab not drawn does not mean the result is zero or normal; it means it was not observed. Reporting "n = 4,200 patients with HbA1c data (mean 8.1%)" without noting that 3,800 patients lacked HbA1c data understates the incompleteness of the measurement.

• Registry: Registries often collect structured clinical variables (stage, performance status, biomarkers) with fewer missing values than EHR or claims, but enrollment is selective. Descriptive statistics from a voluntary registry describe the enrolled population, which may differ systematically from all eligible patients.
• Primary data (surveys, trials): The cleanest setting for descriptive statistics because the denominator is defined by the study design. Report all randomized or consented subjects in the denominator, with a consort-style accounting of how the analysis set differs.
• Linked data: When datasets are linked, report descriptive statistics both for the full eligible population and for the linkable subset, to allow readers to assess linkage-induced selection bias.

Interpreting the output

Consider a Table 1 row for total annual healthcare costs across ten patients in a retrospective claims cohort. The analysis returns: mean = $11,750 (SD ≈ $27,566), median = $2,600 (IQR $1,650–$6,250), n = 10.

(1) Formal statistical interpretation. The mean of $11,750 is the arithmetic average and is sensitive to the single high-cost outlier ($90,000) that pulls it far above most patients' spending. The median of $2,600 splits the ranked distribution into equal halves and is resistant to that extreme value.

The SD of ≈ $27,566 is nearly 2.4 times the mean, signaling severe right-skew; in a skewed distribution the SD is a poor standalone spread summary because it implies symmetry that does not exist. The IQR of $1,650–$6,250 captures the middle 50% of observations and provides an interpretable spread measure that does not assume any particular distributional shape.

(2) Practical interpretation for a decision-maker. The "average" cost of $11,750 is driven almost entirely by one catastrophically ill patient; nine of the ten patients spent under $7,000. For budget modeling and benefit design, the median ($2,600) and IQR better represent what a typical member costs.

When comparing treatment groups, use median and IQR as the primary cost summary and report the mean separately with explicit acknowledgment that it is sensitive to extreme values — a small number of complex cases can pull the group average upward and obscure meaningful differences at the center of the distribution.

Decision diagram

flowchart TD
  Start[Variable to summarize] --> Shape{Inspect histogram and boxplot}
  Shape -->|Approximately symmetric| ContNorm[Report mean and SD<br/>e.g. age, weight, labs]
  Shape -->|Right-skewed<br/>costs, LOS, counts| ContSkew[Report median and IQR<br/>as primary]
  ContSkew --> BudgetQ{Budget impact<br/>needed?}
  BudgetQ -->|Yes| BothMM[Also report mean and SD<br/>mean x n = total plan cost]
  BudgetQ -->|No| MedOnly[Median and IQR alone<br/>sufficient for Table 1]
  Shape -->|Bimodal or multimodal| Stratify[Stratify or plot full distribution<br/>single mean is misleading]
  Start --> CatVar{Categorical variable?}
  CatVar -->|Yes| NPct[n and percent<br/>state denominator explicitly]
  NPct --> Missing[Report missing as<br/>its own n and percent row]

flowchart LR
  subgraph "Table 1 convention"
    A["Normal continuous<br/>e.g. age 62 years (SD 9.3)"]
    B["Skewed continuous<br/>e.g. cost $2,600 [$1,650-$6,250]"]
    C["Binary / categorical<br/>e.g. prior hosp: 40 (35%)"]
    D["Missing<br/>e.g. HbA1c missing: 18 (15%)"]
  end
  A --> |Mean SD| Format1["mean (SD)"]
  B --> |Median IQR| Format2["median [IQR]"]
  B --> |Also needed for budget| Format3["mean (SD) as supplemental"]
  C --> |Count percent| Format4["n (%)"]
  D --> |Never suppress| Format5["n (%) of total cohort"]

Worked example

Scenario

A health economist is characterizing the baseline annual healthcare costs for 10 patients newly enrolled in a commercial claims database. She records the total paid amount (medical + pharmacy combined) for each patient in the 12 months before their index date. She wants to know which summary statistic best represents the typical patient's cost and which best supports a budget impact projection for a plan expecting to enroll 1,000 similar patients.

Dataset

Annual baseline healthcare costs for 10 patients (sorted), 12-month lookback in commercial claims. Patient 10 had a high-cost hospitalization; the other nine had unremarkable utilization.

Steps

Result

Mean cost = $11,750 (SD approximately $27,566); median cost = $2,600 [IQR $1,650-$6,250]. The large gap between mean and median confirms right skew. For Table 1 and clinical characterization, report the median: a typical patient spent $2,600. For budget impact across 1,000 patients, use the mean: projected total plan spend = 1,000 x 11,750 = 11,750,000 dollars. Both summaries are correct -- they answer different questions.

Trade-offs

Runnable example

import numpy as np
import pandas as pd
from scipy.stats import skew


def describe_rwe(df: pd.DataFrame,
                 continuous_cols: list[str],
                 binary_cols: list[str],
                 skew_threshold: float = 1.0) -> pd.DataFrame:
    """
    Table 1-style descriptive statistics for RWE/HEOR datasets.

    Continuous variables:
      - If |skewness| < skew_threshold  -> mean (SD)  [approximately normal]
      - If |skewness| >= skew_threshold -> median [IQR] AND mean (SD)  [HEOR convention for costs]
    Binary variables: n (%)
    Missing values are reported as a separate row for each variable.
    """
    rows = []
    n_total = len(df)

    for col in continuous_cols:
        series = df[col].dropna()
        n_obs = len(series)
        n_miss = n_total - n_obs
        sk = float(skew(series)) if n_obs > 2 else 0.0
        mean_val = float(series.mean())
        sd_val = float(series.std(ddof=1))
        med_val = float(series.median())
        q1 = float(series.quantile(0.25))
        q3 = float(series.quantile(0.75))

        if abs(sk) < skew_threshold:
            # Approximately normal -> report mean (SD) as primary
            rows.append({
                "variable": col,
                "format": "mean (SD)",
                "primary": f"{mean_val:.1f} ({sd_val:.1f})",
                "supplemental": None,
                "n_obs": n_obs,
                "n_missing": n_miss,
                "skewness": round(sk, 2),
            })
        else:
            # Skewed -> report median [IQR] as primary; mean (SD) as supplemental for budget use
            rows.append({
                "variable": col,
                "format": "median [IQR]",
                "primary": f"{med_val:.1f} [{q1:.1f}-{q3:.1f}]",
                "supplemental": f"mean {mean_val:.1f} (SD {sd_val:.1f})",
                "n_obs": n_obs,
                "n_missing": n_miss,
                "skewness": round(sk, 2),
            })

    for col in binary_cols:
        series = df[col]
        n_miss = int(series.isna().sum())
        n_obs = n_total - n_miss
        n_event = int(series.sum()) if n_miss < n_total else 0
        pct = 100.0 * n_event / n_obs if n_obs > 0 else float("nan")
        rows.append({
            "variable": col,
            "format": "n (%)",
            "primary": f"{n_event} ({pct:.1f}%)",
            "supplemental": f"denominator n={n_obs}",
            "n_obs": n_obs,
            "n_missing": n_miss,
            "skewness": None,
        })

    return pd.DataFrame(rows)


# Example for a 10-patient cost dataset
df_example = pd.DataFrame({
    "annual_cost": [1200, 1500, 1800, 2100, 2400, 2800, 3200, 4500, 8000, 90000],
    "age":         [52, 61, 47, 73, 58, 66, 55, 70, 62, 68],
    "prior_hosp":  [0, 0, 0, 1, 0, 0, 1, 0, 1, 1],
})
result = describe_rwe(df_example,
                      continuous_cols=["annual_cost", "age"],
                      binary_cols=["prior_hosp"])
print(result.to_string(index=False))

library(tableone)
library(dplyr)
library(e1071)  # skewness()

# Identify skewed variables automatically (|skewness| >= 1.0 threshold)
detect_skewed <- function(df, continuous_cols, threshold = 1.0) {
  Filter(function(col) {
    vals <- df[[col]][!is.na(df[[col]])]
    abs(e1071::skewness(vals)) >= threshold
  }, continuous_cols)
}

# Build a Table 1 following RWE/HEOR conventions:
#   - non_normal vector -> tableone uses median [IQR]
#   - remaining continuous -> mean (SD)
#   - categorical / binary -> n (%)
#   - test = FALSE: never include p-values in a descriptive Table 1
rwe_table1 <- function(df,
                       continuous_cols,
                       categorical_cols,
                       strata_col = NULL,
                       skew_threshold = 1.0) {

  non_normal <- detect_skewed(df, continuous_cols, skew_threshold)
  all_vars   <- c(continuous_cols, categorical_cols)

  t1 <- CreateTableOne(
    vars       = all_vars,
    strata     = strata_col,
    data       = df,
    factorVars = categorical_cols,
    test       = FALSE  # no p-values in descriptive RWE Table 1
  )
  print(t1,
        nonnormal   = non_normal,  # auto median [IQR]
        smd         = !is.null(strata_col),
        showAllLevels = TRUE,
        quote       = FALSE,
        noSpaces    = TRUE)
}

# Example
df_example <- data.frame(
  annual_cost = c(1200, 1500, 1800, 2100, 2400, 2800, 3200, 4500, 8000, 90000),
  age         = c(52, 61, 47, 73, 58, 66, 55, 70, 62, 68),
  prior_hosp  = factor(c(0, 0, 0, 1, 0, 0, 1, 0, 1, 1), labels = c("No","Yes"))
)
rwe_table1(df_example,
           continuous_cols  = c("annual_cost", "age"),
           categorical_cols = c("prior_hosp"))

# Direct summary for verification
cat("Mean cost:", mean(df_example$annual_cost), "\n")
cat("Median cost:", median(df_example$annual_cost), "\n")
cat("SD cost:", sd(df_example$annual_cost), "\n")
cat("Q1:", quantile(df_example$annual_cost, 0.25), "\n")
cat("Q3:", quantile(df_example$annual_cost, 0.75), "\n")

/* ---- Continuous variables: mean, SD, median, IQR ---- */
/* PROC MEANS: mean, SD, min, max for approximately normal variables (e.g., age) */
proc means data=work.cohort n mean std min max maxdec=1;
  var age;
  title "Continuous variables: mean (SD) for approximately normal";
run;

/* PROC UNIVARIATE: median, Q1, Q3, IQR for skewed variables (costs, LOS, utilization) */
proc univariate data=work.cohort noprint;
  var annual_cost los_days n_outpatient_visits;
  output out=work.cost_stats
    n=n
    mean=mean
    std=std
    median=median
    q1=q1
    q3=q3;
run;

/* Print the descriptive output with IQR computed inline */
data work.cost_stats_iqr;
  set work.cost_stats;
  iqr = q3 - q1;
  /* Table 1 label: median [Q1-Q3] */
  median_iqr = catx(' ', strip(put(median, 8.0)),
                     cats('[', strip(put(q1, 8.0)), '-', strip(put(q3, 8.0)), ']'));
run;

proc print data=work.cost_stats_iqr label noobs;
  var n mean std median iqr median_iqr;
  title "Skewed continuous variables: median [IQR] and mean (SD) -- RWE/HEOR convention";
run;

/* ---- Categorical variables: n (%) with explicit denominator ---- */
/* PROC FREQ with / missing option reports missingness as its own row */
proc freq data=work.cohort;
  tables prior_hosp sex region comorbidity_flag / missing nocum nopercent;
  /* 'missing' option: missing values appear as an explicit row in the output    */
  /* never suppress the missing category when building Table 1 for RWE datasets */
  title "Categorical variables: n (%) -- missing displayed explicitly";
run;

/* ---- Distribution check: skewness and kurtosis to decide mean vs median ---- */
proc univariate data=work.cohort normal;
  var annual_cost age;
  histogram annual_cost / normal;
  qqplot   annual_cost / normal;
  title "Distribution diagnostics: skewness, kurtosis, normality tests";
run;

Citations

1. [1]Lang TA, Altman DG. Basic statistical reporting for articles published in Biomedical Journals: The Statistical Analyses and Methods in the Published Literature or the SAMPL Guidelines. International Journal of Nursing Studies. 2015;52(1):5-9.
2. [2]Altman DG, Bland JM. Statistics Notes: The normal distribution. BMJ. 1995;310(6975):298.
3. [3]Manning WG, Mullahy J. Estimating log models: to transform or not to transform? Journal of Health Economics. 2001;20(4):461-494.

1. [4]Altman DG, Bland JM. Statistics Notes: Quartiles, quintiles, centiles, and other quantiles. BMJ. 1994;309(6960):996.