Descriptive Statistics

In plain language

Descriptive statistics are the arithmetic tools that summarize what a group of patients or a variable actually looks like: the average or typical value (mean and median), how spread out the values are (standard deviation and IQR), and what fraction belong to each category (counts and percentages). They are the foundation every Table 1 is built on and the first step in any data analysis. One key caution: describing a difference between two groups with descriptive statistics does not explain why the difference exists — that requires a separate causal analysis.

Descriptive statistics

answer a single question: "what does this variable look like?" They make no causal claim, test no hypothesis, and require no comparison group. Every downstream analytic step — from a Table 1 to a regression to an economic model — rests on a prior choice about how to summarize the raw distribution, and wrong choices introduce errors that compound through the whole analysis. This entry covers the full toolkit: measures of location, spread, and shape; categorical variable summaries; the Table 1 conventions used in real-world evidence (RWE) and health economics and outcomes research (HEOR); and the most consequential trade-offs, especially the mean-vs-median decision for right-skewed healthcare cost and utilization data.

Core conceptual distinctions

Three questions about any variable answer different questions and require different summaries.

Where is it centered? The arithmetic mean (sum divided by count) is sensitive to every value, including extreme ones. Pull one hospitalization cost to \$500,000 and the mean jumps even if 99 patients had modest bills. The median (middle value after sorting, or the average of the two middle values when n is even) is resistant: it reports what a typical patient experienced. The mode (most frequent value) is rarely used for continuous data but matters for discrete counts and categorical variables.

How spread out is it? The standard deviation (SD) is the square root of the average squared distance from the mean. Because it uses the mean as its anchor, it is equally sensitive to outliers. For a normal distribution, mean ± 1 SD contains approximately 68% of values, mean ± 2 SD contains approximately 95%, and mean ± 3 SD contains approximately 99.7% — the 68-95-99.7 rule. This heuristic fails completely on skewed data: a cost distribution with mean \$12,000 and SD \$28,000 implies a lower bound of mean − 1 SD = −\$16,000, which is meaningless. When data are skewed, report the interquartile range (IQR): the spread from the 25th percentile (Q1) to the 75th percentile (Q3). The IQR describes the middle 50% of the distribution regardless of how extreme the tails are. The range (minimum to maximum) is sensitive to single extreme values and is usually relegated to a footnote. The coefficient of variation (CV) = SD / mean expresses spread as a fraction of the mean — useful when comparing variability across variables measured on different scales.

What shape does it have? Skewness describes asymmetry: a right-skewed (positive-skew) distribution has a long right tail (mean > median), which is the default for healthcare costs, length of stay, and utilization counts. A left-skewed distribution has a long left tail. Kurtosis describes tail weight: high kurtosis ("leptokurtic") means more extreme values than a normal distribution would predict. Concretely, look at a histogram first and a boxplot second — these visual checks tell you whether the normal approximation applies before you decide which summary to use.

The skewed-cost-data rule and the HEOR nuance

The single most important application of this principle in RWE and HEOR is the following: healthcare costs, length of stay, and service-utilization counts are almost always right-skewed because a small fraction of patients generate catastrophically large bills while the majority have modest expenditures. For describing a typical patient's experience, report median and IQR. The mean inflates the central tendency estimate because a handful of \$200,000 admissions pull it far above what most patients incur.

However — and this is a genuinely important nuance for payers and budget impact — the payer pays the mean, not the median. If 1,000 patients are enrolled, the plan's total expenditure is 1,000 × mean, not 1,000 × median. For budget impact analyses, cost-effectiveness models, and any setting that requires total cost estimation, the mean is the appropriate summary because the mean × count = total. For these analyses, report both the mean and median: the median characterizes the typical patient, the mean drives the budget math. When the two diverge substantially, the distribution is skewed enough that the choice of summary matters and should be explained in the methods.

Categorical variables

For binary or multi-category variables, the right summaries are counts (n) and proportions (%). The single most common error is ambiguous denominators: a "30% rate of prior hospitalization" is meaningless without knowing 30% of what — the full cohort? the treated arm? patients with at least 12 months of lookback? Report the denominator explicitly. Missingness is its own category: if 15% of patients lack a recorded HbA1c, that 15% should appear as "missing: n (15%)" in the table, not be silently excluded from the denominator. Collapsing missing patients into a lower denominator inflates the apparent proportion of observed values.

Table 1 conventions in RWE/HEOR

The standard reporting convention for a "Table 1" (baseline characteristics) is: - Approximately normally distributed continuous variables: mean (SD) - Skewed continuous variables (costs, LOS, utilization counts): median [IQR] — many journals use square brackets for IQR to distinguish from the round-bracket mean (SD) convention - Categorical variables: n (%) - For cost variables specifically: consider reporting both mean (SD) and median [IQR] in the same row

Never report standard error of the mean (SEM) in Table 1

The SEM describes the precision of the estimated mean as an estimator — it shrinks as sample size grows. The SD describes the spread of the actual patient population and does not depend on sample size. Table 1 is supposed to show what the patients look like; the SD does that. A 50,000-patient claims cohort with SD = \$28,000 and SEM = \$126 should report SD, because SEM = \$126 falsely implies all patients had near-identical costs. This confusion between SEM and SD is one of the most common errors in reporting of clinical and HEOR data.

Descriptive ≠ inferential: no p-values needed, and why that matters

Descriptive statistics describe. No p-value is needed to state that the mean age is 62 years or that 35% of patients had a prior hospitalization. The long-running debate about whether to include p-values in Table 1 of a randomized trial (testing whether randomization succeeded) does not apply to RWE: in a non-randomized study, a significant p-value on a baseline covariate is expected (treatment was chosen, not assigned) and uninformative. Whether to worry about a 3-year age difference or a 0.5-year difference depends on whether age is a confounder for the outcome of interest — a clinical judgment, not a significance test. In non-randomized RWE, standardized mean differences (SMDs) replace p-values for assessing whether groups are comparable (see baseline-characteristics-and-covariate-balance-rwe).

Pros, cons, and trade-offs — specific and comparative

• Mean vs median: The mean is the natural input to budget math (mean × n = total), and means are
• SD vs IQR: The SD is the natural companion to the mean and is required when computing confidence
• SEM vs SD (the cardinal confusion): The SEM is used in confidence intervals for the mean, not in
• Descriptive vs causal summaries: Descriptive statistics are robust precisely because they make no
• Pooling vs stratifying: Summarizing a bimodal or multimodal distribution with a single mean hides

When to use

Descriptive statistics should be the first analytic step in any study: - Characterize the study population in Table 1 (demographics, comorbidities, baseline HCRU, costs) before any outcome model is fit. - Understand the distribution of each variable — mean, median, SD, IQR, min, max, histogram — before deciding which analytical method is appropriate (parametric vs nonparametric, log-transformation vs GLM with gamma link for costs, etc.). - Report frequencies and proportions for all categorical variables including missingness. - Provide context for quantitative findings (e.g., "the median cost was \$X versus \$Y in the comparator" alongside an adjusted ratio). - Conduct feasibility and sample-size exploration: understanding effect sizes, event rates, and variances from descriptive statistics feeds power calculations.

When NOT to use — and when descriptive summaries are actively misleading

• Do not use descriptive between-group differences to support causal claims. A mean cost difference
• Do not report means on heavily right-skewed data without also reporting the median. When the mean
• Do not report means on censored cost data without appropriate adjustment. When follow-up time is
• Do not pool a bimodal or multimodal distribution into one mean. If a variable has two distinct
• Do not confuse SEM with SD in Table 1 — this is the most common descriptive statistics error in
• Do not describe with ambiguous denominators. Any percentage must specify what the denominator

Data-source operational depth

• Claims (FFS): Most continuous variables of interest — costs, length of stay, service counts —
• EHR: Laboratory values, vitals, and clinical scores are often closer to normal than cost data,
• Registry: Registries often collect structured clinical variables (stage, performance status,
• Primary data (surveys, trials): The cleanest setting for descriptive statistics because the
• Linked data: When datasets are linked, report descriptive statistics both for the full eligible

Interpreting the output

Consider a Table 1 row for total annual healthcare costs across ten patients in a retrospective claims cohort. The analysis returns: mean = $11,750 (SD ≈ $27,566), median = $2,600 (IQR $1,650–$6,250), n = 10.

(1) Formal statistical interpretation. The mean of $11,750 is the arithmetic average and is sensitive to the single high-cost outlier ($90,000) that pulls it far above most patients' spending. The median of $2,600 splits the ranked distribution into equal halves and is resistant to that extreme value. The SD of ≈ $27,566 is nearly 2.4 times the mean, signaling severe right-skew; in a skewed distribution the SD is a poor standalone spread summary because it implies symmetry that does not exist. The IQR of $1,650–$6,250 captures the middle 50% of observations and provides an interpretable spread measure that does not assume any particular distributional shape.

(2) Practical interpretation for a decision-maker. The "average" cost of $11,750 is driven almost entirely by one catastrophically ill patient; nine of the ten patients spent under $7,000. For budget modeling and benefit design, the median ($2,600) and IQR better represent what a typical member costs. When comparing treatment groups, use median and IQR as the primary cost summary and report the mean separately with explicit acknowledgment that it is sensitive to extreme values — a small number of complex cases can pull the group average upward and obscure meaningful differences at the center of the distribution.

Worked example

Scenario

A health economist is characterizing the baseline annual healthcare costs for 10 patients newly enrolled in a commercial claims database. She records the total paid amount (medical + pharmacy combined) for each patient in the 12 months before their index date. She wants to know which summary statistic best represents the typical patient's cost and which best supports a budget impact projection for a plan expecting to enroll 1,000 similar patients.

Dataset

Annual baseline healthcare costs for 10 patients (sorted), 12-month lookback in commercial claims. Patient 10 had a high-cost hospitalization; the other nine had unremarkable utilization.

Steps

• Compute the mean: sum all 10 values, then divide by 10. Sum = 1200 + 1500 + 1800 + 2100 + 2400 + 2800 + 3200 + 4500 + 8000 + 90000 = 117500. Mean = 117500 / 10 = 11750.

• Compute the median: with 10 values already sorted, the median is the average of the 5th and 6th values. The 5th value is 2400 and the 6th is 2800. Median = (2400 + 2800) / 2 = 2600.

• Note the gap: the mean ($11,750) is more than four times the median ($2,600). That gap signals strong right skew driven by the one $90,000 outlier patient.

• Compute Q1 (25th percentile) and Q3 (75th percentile) for the IQR. With 10 sorted values, Q1 is the average of the 2nd and 3rd values and Q3 is the average of the 8th and 9th values. Q1 = (1500 + 1800) / 2 = 1650. Q3 = (4500 + 8000) / 2 = 6250. IQR = 6250 - 1650 = 4600.

• Interpret the two summaries: the median $2,600 with IQR $1,650 to $6,250 describes the typical patient experience accurately -- nine of ten patients spent between $1,200 and $8,000. The mean $11,750 looks like an outlier relative to most patients, but it is the correct input for budget projections: 1,000 enrolled patients would cost the plan approximately 1,000 x 11,750 = 11,750,000 dollars, not 1,000 x 2,600 = 2,600,000 dollars.

• Table 1 reporting convention: for this right-skewed cost variable, report median [IQR] to characterize the typical patient: $2,600 [$1,650-$6,250]. For the budget model, use the mean: $11,750.

Result

Mean cost = $11,750 (SD approximately $27,566); median cost = $2,600 [IQR $1,650-$6,250]. The large gap between mean and median confirms right skew. For Table 1 and clinical characterization, report the median: a typical patient spent $2,600. For budget impact across 1,000 patients, use the mean: projected total plan spend = 1,000 x 11,750 = 11,750,000 dollars. Both summaries are correct -- they answer different questions.

Runnable example

import numpy as np
import pandas as pd
from scipy.stats import skew


def describe_rwe(df: pd.DataFrame,
                 continuous_cols: list[str],
                 binary_cols: list[str],
                 skew_threshold: float = 1.0) -> pd.DataFrame:
    """
    Table 1-style descriptive statistics for RWE/HEOR datasets.

    Continuous variables:
      - If |skewness| < skew_threshold  -> mean (SD)  [approximately normal]
      - If |skewness| >= skew_threshold -> median [IQR] AND mean (SD)  [HEOR convention for costs]
    Binary variables: n (%)
    Missing values are reported as a separate row for each variable.
    """
    rows = []
    n_total = len(df)

    for col in continuous_cols:
        series = df[col].dropna()
        n_obs = len(series)
        n_miss = n_total - n_obs
        sk = float(skew(series)) if n_obs > 2 else 0.0
        mean_val = float(series.mean())
        sd_val = float(series.std(ddof=1))
        med_val = float(series.median())
        q1 = float(series.quantile(0.25))
        q3 = float(series.quantile(0.75))

        if abs(sk) < skew_threshold:
            # Approximately normal -> report mean (SD) as primary
            rows.append({
                "variable": col,
                "format": "mean (SD)",
                "primary": f"{mean_val:.1f} ({sd_val:.1f})",
                "supplemental": None,
                "n_obs": n_obs,
                "n_missing": n_miss,
                "skewness": round(sk, 2),
            })
        else:
            # Skewed -> report median [IQR] as primary; mean (SD) as supplemental for budget use
            rows.append({
                "variable": col,
                "format": "median [IQR]",
                "primary": f"{med_val:.1f} [{q1:.1f}-{q3:.1f}]",
                "supplemental": f"mean {mean_val:.1f} (SD {sd_val:.1f})",
                "n_obs": n_obs,
                "n_missing": n_miss,
                "skewness": round(sk, 2),
            })

    for col in binary_cols:
        series = df[col]
        n_miss = int(series.isna().sum())
        n_obs = n_total - n_miss
        n_event = int(series.sum()) if n_miss < n_total else 0
        pct = 100.0 * n_event / n_obs if n_obs > 0 else float("nan")
        rows.append({
            "variable": col,
            "format": "n (%)",
            "primary": f"{n_event} ({pct:.1f}%)",
            "supplemental": f"denominator n={n_obs}",
            "n_obs": n_obs,
            "n_missing": n_miss,
            "skewness": None,
        })

    return pd.DataFrame(rows)


# Example for a 10-patient cost dataset
df_example = pd.DataFrame({
    "annual_cost": [1200, 1500, 1800, 2100, 2400, 2800, 3200, 4500, 8000, 90000],
    "age":         [52, 61, 47, 73, 58, 66, 55, 70, 62, 68],
    "prior_hosp":  [0, 0, 0, 1, 0, 0, 1, 0, 1, 1],
})
result = describe_rwe(df_example,
                      continuous_cols=["annual_cost", "age"],
                      binary_cols=["prior_hosp"])
print(result.to_string(index=False))

library(tableone)
library(dplyr)
library(e1071)  # skewness()

# Identify skewed variables automatically (|skewness| >= 1.0 threshold)
detect_skewed <- function(df, continuous_cols, threshold = 1.0) {
  Filter(function(col) {
    vals <- df[[col]][!is.na(df[[col]])]
    abs(e1071::skewness(vals)) >= threshold
  }, continuous_cols)
}

# Build a Table 1 following RWE/HEOR conventions:
#   - non_normal vector -> tableone uses median [IQR]
#   - remaining continuous -> mean (SD)
#   - categorical / binary -> n (%)
#   - test = FALSE: never include p-values in a descriptive Table 1
rwe_table1 <- function(df,
                       continuous_cols,
                       categorical_cols,
                       strata_col = NULL,
                       skew_threshold = 1.0) {

  non_normal <- detect_skewed(df, continuous_cols, skew_threshold)
  all_vars   <- c(continuous_cols, categorical_cols)

  t1 <- CreateTableOne(
    vars       = all_vars,
    strata     = strata_col,
    data       = df,
    factorVars = categorical_cols,
    test       = FALSE  # no p-values in descriptive RWE Table 1
  )
  print(t1,
        nonnormal   = non_normal,  # auto median [IQR]
        smd         = !is.null(strata_col),
        showAllLevels = TRUE,
        quote       = FALSE,
        noSpaces    = TRUE)
}

# Example
df_example <- data.frame(
  annual_cost = c(1200, 1500, 1800, 2100, 2400, 2800, 3200, 4500, 8000, 90000),
  age         = c(52, 61, 47, 73, 58, 66, 55, 70, 62, 68),
  prior_hosp  = factor(c(0, 0, 0, 1, 0, 0, 1, 0, 1, 1), labels = c("No","Yes"))
)
rwe_table1(df_example,
           continuous_cols  = c("annual_cost", "age"),
           categorical_cols = c("prior_hosp"))

# Direct summary for verification
cat("Mean cost:", mean(df_example$annual_cost), "\n")
cat("Median cost:", median(df_example$annual_cost), "\n")
cat("SD cost:", sd(df_example$annual_cost), "\n")
cat("Q1:", quantile(df_example$annual_cost, 0.25), "\n")
cat("Q3:", quantile(df_example$annual_cost, 0.75), "\n")