Wilcoxon Signed-Rank Test

What the Wilcoxon signed-rank test is and what it does

The Wilcoxon signed-rank test, introduced by Frank Wilcoxon in 1945, tests whether the median (more precisely, the pseudomedian) of paired differences is zero. It is the standard nonparametric alternative to the paired t-test when the distribution of within-pair differences is non-normal or when sample size is too small for the Central Limit Theorem to protect inference on the mean difference. The test operates on the paired differences themselves — not on the raw observations — and constructs its test statistic from the ranks of those differences' absolute values.

The mechanics proceed in four steps. First, compute the signed difference d_i = x_i2 - x_i1 for each pair i. Second, rank the absolute values |d_i| from smallest (rank 1) to largest (rank n), ignoring any pair with d_i = 0 by the standard Wilcoxon convention (more on this below). Third, reattach the sign of each original difference to its rank, yielding signed ranks. Fourth, compute W+ (sum of positive signed ranks) and W- (sum of negative signed ranks). The test statistic is conventionally reported as W = min(W+, W-). Under the null hypothesis that differences are symmetrically distributed around zero, W+ and W- should each be close to n(n+1)/4, and their total W+ + W- = n(n+1)/2 always holds as a useful arithmetic check.

The symmetry assumption — the one that often goes unacknowledged

The Wilcoxon signed-rank test is frequently described as "assumption-free" or as requiring only that the differences be independent. This is incorrect and the misunderstanding has real consequences in practice. The test requires that the distribution of differences be symmetric around its median, not merely that it be continuous. Under the null hypothesis the symmetry assumption is automatically satisfied (a distribution symmetric around zero satisfies any symmetry claim), but the test's power properties and interpretation under the alternative depend on symmetry holding. When the difference distribution is highly skewed — for example, most patients improve modestly and a few improve dramatically — the signed-rank test can reject for a reason other than a shift in location, because skewness itself disrupts the rank-sign balance the test exploits.

In RWE applications, cost changes (post - pre) are often right-skewed: most patients have moderate cost increases and a few have catastrophic spikes. Under this asymmetry, a significant Wilcoxon result does not cleanly identify a median shift — it may be detecting the skewness. The safe interpretation in this setting is that the two measurement occasions produce different rank patterns, not necessarily a symmetric shift in location.

Zeros and ties: the Pratt versus Wilcoxon method

In real data, some paired differences will be exactly zero (no change). The original Wilcoxon (1945) convention discards zero differences and reduces n accordingly. An alternative due to Pratt (1959) retains all pairs — zero differences are ranked but their contribution to W+ and W- cancels (they carry rank but no sign). In scipy.stats.wilcoxon, the `zero_method` argument controls this: `zero_method='wilcoxon'` (the default) drops zeros; `zero_method='pratt'` retains them. In R's wilcox.test(paired=TRUE), zeros are dropped by default. The choice matters when the fraction of zeros is non-trivial: dropping zeros reduces n and can inflate power if many zeros exist because of a true null; Pratt's method avoids this inflation. Document which method was used in the analysis SAP.

Ties in the absolute differences (two pairs with the same |d|) are handled by averaging the ranks they would occupy, exactly as in the Mann-Whitney U test. Large numbers of ties reduce the information content of the ranks; when ties are extensive (> 20% of pairs) the normal approximation for the p-value uses a tie-corrected variance. Exact p-values are available in small samples and are recommended when n < 25; for n ≥ 25 the normal approximation is adequate unless ties are heavy.

The pseudomedian and the Hodges-Lehmann estimator for the Wilcoxon signed-rank test

A key practical point that is frequently overlooked: the effect estimate that pairs naturally with the Wilcoxon signed-rank test is the pseudomedian of the differences — not the median of the differences, and not the difference of the medians. The pseudomedian of a sample is the median of all pairwise averages (d_i + d_j)/2 for i ≤ j, including d_i with itself (the Walsh averages). For a symmetric distribution the pseudomedian equals the median, but for a skewed difference distribution the two can diverge substantially. The Hodges-Lehmann estimator implements the pseudomedian and is the standard companion estimate for the Wilcoxon signed-rank test; in R, wilcox.test(paired=TRUE, conf.int=TRUE) returns it automatically. Reporting only a Wilcoxon p-value without this effect estimate leaves the reader unable to assess the magnitude or direction of the change.

The distinction matters operationally: if the analysis question is "did patients improve on average?" the mean difference (paired t-test or a mixed model) is the right target. If the question is "did a typical patient improve?" the median or pseudomedian of the differences is appropriate. In HEOR, cost-effectiveness models and budget-impact analyses need the mean change, not the pseudomedian, so the Wilcoxon signed-rank test is usually a sensitivity check alongside a model-based mean-change estimator for cost outcomes.

The sign test as the weaker, assumption-free fallback

When the symmetry assumption for the signed-rank test is clearly violated and the analyst wants a strictly assumption-free test for paired data, the sign test is the fallback. The sign test counts only whether each d_i is positive or negative (discarding magnitude entirely) and tests whether positive differences are equally likely as negative ones under the null. Because it discards rank information, the sign test is markedly less powerful than the Wilcoxon signed-rank test; it should be reserved for situations where the symmetry assumption is demonstrably implausible and the analyst cannot use a model-based approach.

RWE and HEOR applications

Pre-post healthcare utilization and costs: When the same patients are observed before and after a treatment initiation, procedure, or policy change and the outcome is an inherently skewed measure (emergency department visits, total allowed costs, inpatient days), the paired t-test on the within-patient difference can be distorted by outliers. The Wilcoxon signed-rank test provides a robust sensitivity check. However, as noted above, if the mean cost change is the decision-relevant quantity (for a budget-impact model), the Wilcoxon test is supplementary — a gamma GLM or bootstrap mean-difference estimator should be the primary analysis.

Patient-reported outcomes (PROs) and utility scores: EQ-5D utility scores, SF-6D scores, and disease-specific PROs often violate normality: they are bounded (0 to 1 or 0 to 100), may cluster near boundary values, and have pre-post differences that are neither continuous nor bell-shaped. The Wilcoxon signed-rank test is the standard nonparametric paired test in this context and is frequently required alongside the parametric test in health technology assessment (HTA) submissions.

Matched cohort designs: When 1:1 matched pairs are constructed in an observational study (propensity-score matched, exact-matched, or coarsened exact matched), the paired structure must be respected. A two-sample t-test or Mann-Whitney on matched pairs ignores the pairing and loses efficiency. The Wilcoxon signed-rank test correctly exploits the paired design by operating on within-pair differences, exactly as in the pre-post setting.

Pseudomedian-of-differences ≠ difference-of-medians — why this matters

In practice, analysts sometimes report the median change as "median(after) - median(before)" which is the difference of two group medians. This is not what the Wilcoxon signed-rank test tests, and it is not the pseudomedian of the differences. Consider a simple example: if half the patients improve from 4 to 8 (+4) and half improve from 5 to 9 (+4), both medians shift by exactly 4 and the pseudomedian of differences is 4. But if pre-scores are 4 and 5 (median 4.5) and post-scores are 8 and 9 (median 8.5), then the difference of medians is 4.0 and the pseudomedian is also 4.0 — they agree. However, in skewed real data they can diverge. The correct companion to the Wilcoxon test is always the Hodges-Lehmann pseudomedian; report it.

Pros, cons, and trade-offs

Pros:

• Valid when the paired t-test's normality assumption is violated and n is small.
• Robust to outliers in the differences: an extreme outlier contributes at most rank n rather than distorting the mean as it would in a paired t-test.
• Exact p-values are available for small n (< 25), making it reliable in pilot or registry sub-studies with limited sample sizes.
• The Hodges-Lehmann pseudomedian with confidence interval is a principled effect estimate that pairs naturally with the test.
• Widely expected by regulators (FDA, EMA, NICE) as a sensitivity check alongside a parametric primary analysis.

Cons:

• The symmetry assumption is often unacknowledged and can be violated in practice.
• Lower power than the paired t-test when the normality assumption holds (the paired t-test uses the actual magnitude of differences, the signed-rank test uses only their ranks).
• The pseudomedian is not the effect estimate that budget-impact or cost-effectiveness models require — they need the mean change.
• Cannot be directly extended to covariate adjustment. Regression models on the within-pair differences (ANCOVA or a mixed model) are needed when baseline imbalance or confounders must be controlled.
• Zero differences require a decision (Wilcoxon vs Pratt method) that should be pre-specified in the SAP.

Trade-offs vs alternatives:

• Against the paired t-test: prefer the Wilcoxon signed-rank test when n < 30 and the difference distribution is clearly non-normal or when outliers are expected; prefer the paired t-test (or a mixed model) when the mean change is the target estimand.
• Against the sign test: the Wilcoxon signed-rank test is almost always more powerful than the sign test; use the sign test only if the symmetry assumption is clearly untenable.
• Against a regression/GLM on differences: when covariate adjustment is needed or the outcome is a count or cost, a regression approach (Poisson, gamma, or negative binomial with a paired-design term) is superior to either the Wilcoxon test or the paired t-test.

When NOT to use

• Independent groups: the Wilcoxon signed-rank test requires paired data; for two independent groups use the Mann-Whitney U test (Wilcoxon rank-sum).
• When the mean change is the estimand: if the result will feed a cost-effectiveness model, budget-impact model, or any analysis that aggregates to population totals, the mean change is needed, not the pseudomedian. Use the paired t-test, a GLM on differences, or a bootstrap mean estimator for the primary analysis.
• Count or zero-inflated pre-post outcomes: for outcomes like number of hospitalizations or number of prescriptions (discrete counts, often with many zeros), a model-based approach (negative binomial or hurdle model with individual as random effect) is more appropriate than a rank test, because the rank test cannot account for the data-generating mechanism.
• More than two timepoints: the Wilcoxon signed-rank test compares a single pair. For repeated measures across more than two time points use a linear mixed model, a Friedman test (the nonparametric k-sample extension), or a GEE.
• When covariate adjustment is needed: rank tests cannot incorporate baseline covariates or confounders. Route to ANCOVA, a mixed effects model on differences, or propensity-score methods if adjustment is required before or after the paired comparison.
• When the symmetry assumption is demonstrably violated and the sample is small: if the difference distribution is known to be highly skewed (e.g., costs, claims counts) and n is large enough that a model-based approach is feasible, prefer a GLM over the signed-rank test, because the test may detect skewness rather than a location shift.

Interpreting the output

In the worked example, six pain patients were assessed before and after treatment. The sum of positive signed ranks is W+ = 15 and the sum of negative signed ranks is W− = 6; the check W+ + W− = 21 = 6 × 7 / 2 confirms the arithmetic. The test statistic is W = min(15, 6) = 6. The exact two-sided p-value at n = 6 is approximately 0.22. The Hodges-Lehmann pseudomedian of the within-patient differences is approximately 2.5 pain-score points of reduction.

(1) Formal interpretation. Under the null hypothesis that the paired differences are symmetrically distributed around zero, a W statistic of 6 or smaller in either tail has a probability of approximately 0.22 — well above alpha = 0.05. The test does not reject the null. At n = 6, the exact distribution of W has very limited resolution: only a small number of rank arrangements exist, so the test has low power to detect moderate effects. The pseudomedian of approximately 2.5 points indicates a positive directional trend, but the sample is far too small to confirm the signal statistically.

(2) Practical interpretation. All six patients showed some reduction in pain, and the Hodges-Lehmann pseudomedian of approximately 2.5 points is the appropriate effect estimate to report alongside this test — it is the pseudomedian of the within-patient differences, not the difference between the pre- and post-period medians. The non-significant p-value at n = 6 is better read as "inconclusive" than "no effect." The symmetry assumption required by the signed-rank test should be assessed; if the difference distribution is strongly skewed (most patients improve slightly, a few improve dramatically), a model-based analysis may be more appropriate, as the signed-rank test can detect skewness rather than a pure location shift.

Decision diagram

flowchart TD
  Start["Paired / pre-post<br/>continuous or ordinal outcome?"] --> CheckN{n pairs<br/>< 30?}
  CheckN -- "Yes, small n" --> CheckNorm{Differences<br/>clearly non-normal<br/>or outliers expected?}
  CheckN -- "No, large n" --> CheckEstimand{Target estimand?}
  CheckNorm -- "Yes" --> WSR["Wilcoxon signed-rank test<br/>(exact p-value)<br/>+ Hodges-Lehmann CI"]
  CheckNorm -- "No" --> PairedT["Paired t-test<br/>(mean difference + CI)"]
  CheckEstimand -- "Pseudomedian<br/>(typical patient)" --> WSRLarge["Wilcoxon signed-rank<br/>(normal approx)<br/>+ Hodges-Lehmann CI"]
  CheckEstimand -- "Mean change<br/>(budget / HTA)" --> Model["Paired t-test or<br/>GLM on differences<br/>(gamma / NB for costs)"]
  WSR --> Adjust{Covariate<br/>adjustment<br/>needed?}
  PairedT --> Adjust
  WSRLarge --> Adjust
  Model --> Done["Report estimate + CI<br/>+ p-value"]
  Adjust -- "No" --> Done
  Adjust -- "Yes" --> ANCOVA["ANCOVA or mixed model<br/>on differences<br/>(rank tests can't adjust)"]
  ANCOVA --> Done

Worked example

Scenario

A HEOR analyst is evaluating a 12-week pain management program in six patients. Each patient rates their pain on a 0-to-10 scale at baseline (week 0) and at week 12. The analyst wants to test whether pain scores changed significantly, but with only six patients the normality assumption for a paired t-test cannot be verified, so the Wilcoxon signed-rank test is chosen. All six differences have distinct absolute values and none equals zero, giving a clean illustration of the core mechanics.

Dataset

Pain scale ratings (0 = no pain, 10 = worst pain) for six patients before and after a 12-week pain management program. Difference = baseline score minus week-12 score; a positive difference means improvement (pain decreased).

Steps

Result

W+ = 5 + 3 + 6 + 1 = 15, W- = 2 + 4 = 6, W+ + W- = 15 + 6 = 21 = 6*7/2 = 21 (check passes). Test statistic W = min(15, 6) = 6. Four of six patients improved (positive differences) and those improvements carried the larger ranks (5, 3, 6, 1), while the two worsening patients had smaller absolute ranks (2, 4). The pseudomedian of the six differences is the median of all 21 Walsh averages; the Hodges-Lehmann estimate equals 2.5 points of pain reduction. Exact two-sided p ≈ 0.22 — not significant at alpha = 0.05 with n = 6, illustrating that small samples require larger effect sizes to achieve significance.

Trade-offs

Runnable example

from scipy import stats
import itertools

# ── Data: six paired pain scores (baseline minus week-12, positive = improvement) ──
baseline = [7, 6, 4, 8, 5, 3]
week12   = [2, 3, 6, 2, 4, 7]
diffs = [b - a for b, a in zip(baseline, week12)]
# diffs = [5, 3, -2, 6, 1, -4]  -> 4 positive, 2 negative

# ── 1. Wilcoxon signed-rank test (default: zero_method='wilcoxon', exact p-value at small n) ──
# Note: scipy.stats.wilcoxon takes the differences directly (or x and y separately).
w_stat, p_val = stats.wilcoxon(diffs, zero_method='wilcoxon', mode='auto')
print(f"Wilcoxon signed-rank test:")
print(f"  W statistic (sum of positive ranks) = {w_stat:.1f}")
print(f"  Two-sided p-value = {p_val:.4f}")
print(f"  (With n=6 scipy uses exact enumeration; normal approximation used for n >= 25)")

# ── 2. Verify W+ and W- by hand ──
abs_diffs = [abs(d) for d in diffs]
# Sort indices by absolute value to assign ranks
sorted_idx = sorted(range(len(abs_diffs)), key=lambda i: abs_diffs[i])
ranks = [0] * len(diffs)
for rank_pos, orig_idx in enumerate(sorted_idx):
    ranks[orig_idx] = rank_pos + 1   # ranks are 1-indexed
W_plus  = sum(r for d, r in zip(diffs, ranks) if d > 0)
W_minus = sum(r for d, r in zip(diffs, ranks) if d < 0)
n = len(diffs)
print(f"\nManual verification:")
print(f"  W+ (sum of positive ranks) = {W_plus}")
print(f"  W- (sum of negative ranks) = {W_minus}")
print(f"  W+ + W- = {W_plus + W_minus}  (should equal n*(n+1)/2 = {n*(n+1)//2})")
print(f"  W = min(W+, W-) = {min(W_plus, W_minus)}")

# ── 3. Pratt method (retains zero differences, assigns ranks but no sign contribution) ──
# Not relevant here (no zeros), but shown for completeness
diffs_with_zero = diffs + [0]   # artificial example with a zero
w_pratt, p_pratt = stats.wilcoxon(diffs_with_zero, zero_method='pratt', mode='approx')
print(f"\nPratt method example (n=7 including one zero):")
print(f"  W = {w_pratt:.1f}, p (normal approx) = {p_pratt:.4f}")

# ── 4. Hodges-Lehmann pseudomedian: median of all Walsh averages (d_i + d_j)/2 for i <= j ──
# This is the correct companion effect estimate for the Wilcoxon signed-rank test.
n = len(diffs)
walsh = [(diffs[i] + diffs[j]) / 2 for i in range(n) for j in range(i, n)]
walsh.sort()
nw = len(walsh)
if nw % 2 == 1:
    pseudomedian = walsh[nw // 2]
else:
    pseudomedian = (walsh[nw // 2 - 1] + walsh[nw // 2]) / 2
print(f"\nHodges-Lehmann pseudomedian of differences: {pseudomedian:.2f}")
print(f"  (Number of Walsh averages = n*(n+1)/2 = {n*(n+1)//2}, computed {nw})")
print(f"  Pseudomedian ≠ simple median of differences ({sorted(diffs)[nw//2 - 1]:.1f})")
print("  Always report this estimate with a CI — p-value alone does not convey effect size.")

# ── 5. Normal approximation for large n ──
import math
n_large = 50  # example large sample
w_example = 450  # hypothetical W+ statistic
mu_w = n_large * (n_large + 1) / 4
sigma_w = math.sqrt(n_large * (n_large + 1) * (2 * n_large + 1) / 24)
z = (w_example - mu_w - 0.5) / sigma_w   # continuity correction of 0.5
from scipy.stats import norm
p_approx = 2 * norm.sf(abs(z))
print(f"\nNormal approximation (n=50, W+={w_example}):")
print(f"  mu_W = {mu_w:.1f}, sigma_W = {sigma_w:.3f}")
print(f"  z = ({w_example} - {mu_w:.1f} - 0.5) / {sigma_w:.3f} = {z:.3f}")
print(f"  Two-sided p (normal approx) = {p_approx:.4f}")

# ── Data: six paired pain scores ──
baseline <- c(7, 6, 4, 8, 5, 3)
week12   <- c(2, 3, 6, 2, 4, 7)
diffs    <- baseline - week12   # c(5, 3, -2, 6, 1, -4)

# ── 1. Wilcoxon signed-rank test with Hodges-Lehmann pseudomedian and CI ──
# conf.int=TRUE returns the Hodges-Lehmann estimate and 95% CI automatically.
# R drops zero differences by default (Wilcoxon convention).
# exact=TRUE uses exact enumeration (recommended at n < 25).
wsr_res <- wilcox.test(baseline, week12, paired = TRUE,
                       conf.int = TRUE, exact = TRUE)
cat("Wilcoxon signed-rank test:\n")
print(wsr_res)
# The output includes:
#   V: the W+ statistic (sum of positive signed ranks); R calls it V
#   p-value: exact two-sided p-value
#   pseudomedian: Hodges-Lehmann estimate
#   conf.int: 95% CI for the pseudomedian
cat(sprintf("\nHodges-Lehmann pseudomedian = %.2f\n", wsr_res$estimate))
cat(sprintf("95%% CI: [%.2f, %.2f]\n", wsr_res$conf.int[1], wsr_res$conf.int[2]))
cat("Note: R labels W+ as 'V'; scipy labels it 'statistic'. Both equal sum of positive ranks.\n")

# ── 2. Manual W+ / W- calculation to match worked example ──
abs_d <- abs(diffs)
ranks <- rank(abs_d)   # R's rank() handles ties by averaging
W_plus  <- sum(ranks[diffs > 0])
W_minus <- sum(ranks[diffs < 0])
n <- length(diffs)
cat(sprintf("\nManual verification:\n  W+ = %.0f,  W- = %.0f\n", W_plus, W_minus))
cat(sprintf("  W+ + W- = %.0f  (should be n*(n+1)/2 = %d)\n", W_plus + W_minus, n*(n+1)/2))
cat(sprintf("  W = min(W+, W-) = %.0f\n", min(W_plus, W_minus)))

# ── 3. Normal approximation for large n (exact=FALSE) ──
# Demonstrate on a larger hypothetical dataset
set.seed(42)
large_pre  <- round(runif(60, 4, 9))
large_post <- round(runif(60, 2, 8))
wsr_large <- wilcox.test(large_pre, large_post, paired = TRUE,
                         conf.int = TRUE, exact = FALSE)
cat("\nLarge sample example (n=60, normal approximation):\n")
print(wsr_large)

# ── 4. Paired t-test for comparison ──
# Show that both tests agree in direction but may differ in p-value
t_res <- t.test(baseline, week12, paired = TRUE)
cat(sprintf("\nPaired t-test for comparison:\n  t = %.3f,  p = %.4f\n",
            t_res$statistic, t_res$p.value))
cat(sprintf("  Mean difference = %.2f (95%% CI: [%.2f, %.2f])\n",
            t_res$estimate, t_res$conf.int[1], t_res$conf.int[2]))
cat("Note: At n=6, both tests are underpowered; with this data neither reaches p < 0.05.\n")

/* ── Create the six-patient paired dataset ── */
data work.pain;
  input patient_id $ baseline week12;
  diff = baseline - week12;   /* positive = improvement */
  datalines;
P1 7 2
P2 6 3
P3 4 6
P4 8 2
P5 5 4
P6 3 7
;
run;

/* ── Verify differences ── */
proc means data=work.pain n mean std min max;
  var diff;
  title "Within-patient pain differences (baseline - week12)";
run;

/* ── Wilcoxon signed-rank test via PROC UNIVARIATE ──
   UNIVARIATE tests H0: median(diff) = 0 using the Wilcoxon signed-rank S statistic.
   The S statistic = W+ - n*(n+1)/4 (centered; SAS convention).
   The p-value for the signed-rank test appears in the "Tests for Location" table.
   CIPCTLDF gives distribution-free CIs (Hodges-Lehmann); CIBASIC gives normal-based CIs. */
proc univariate data=work.pain cipctldf cibasic;
  var diff;
  /* Key output: "Tests for Location" table; look for:
     Signed Rank  S = <value>  Pr >= |S| = <p-value>              */
  title "Wilcoxon Signed-Rank Test on Pain Differences";
run;

/* ── Paired t-test for comparison via PROC TTEST ──
   PROC TTEST with PAIRED statement computes the paired t-test directly from the
   two raw variables; no need to compute differences manually.                     */
proc ttest data=work.pain;
  paired baseline * week12;
  title "Paired t-test on Pain Scores (parametric comparison)";
run;

/* ── Larger example using PROC UNIVARIATE with normal approximation ──
   For n >= 25, the signed-rank statistic is approximately normally distributed.
   SAS automatically uses the correct approximation.                               */
data work.pain_large;
  call streaminit(42);
  do i = 1 to 60;
    patient_id = i;
    baseline = round(4 + rand('uniform') * 5);
    week12   = round(2 + rand('uniform') * 6);
    diff = baseline - week12;
    output;
  end;
  drop i;
run;

proc univariate data=work.pain_large cipctldf;
  var diff;
  title "Wilcoxon Signed-Rank Test — Large Sample (n=60)";
run;

/* Note: SAS PROC NPAR1WAY with the WILCOXON option tests TWO INDEPENDENT groups
   (equivalent to Mann-Whitney U). Do NOT use NPAR1WAY for paired data — use
   PROC UNIVARIATE on the within-patient differences instead.                     */

Citations

1. [1]Wilcoxon F. Individual comparisons by ranking methods. Biometrics Bulletin. 1945;1(6):80-83.
2. [2]Conover WJ, Iman RL. Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician. 1981;35(3):124-129.
3. [3]Nachar N. The Mann-Whitney U: A test for assessing whether two independent samples come from the same distribution. Tutorials in Quantitative Methods for Psychology. 2008;4(1):13-20.

1. [4]Fagerland MW. t-tests, non-parametric tests, and large studies — a paradox of statistical practice? BMC Medical Research Methodology. 2012;12:78.

1. [5]Vickers AJ. Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Medical Research Methodology. 2005;5:35.