Wilcoxon Signed-Rank Test

In plain language

The Wilcoxon signed-rank test answers the question "did the same patients change between two time points?" without assuming their measurements follow a bell-curve distribution. It works by computing each patient's before-minus-after difference, ranking those differences from smallest to largest by size (ignoring direction for now), and then checking whether the big ranks tend to cluster on the positive or negative side. Four verified DOIs anchor the entry: Wilcoxon 1945 (original method), Conover & Iman 1981 (rank-based perspective), Nachar 2008 (practical guide), and Fagerland 2012 (large-sample paradox). One important caveat: the test is not assumption-free — it assumes the distribution of differences is roughly symmetric, so it can give misleading results when skewed outcomes like healthcare costs are involved.

What the Wilcoxon signed-rank test is and what it does

The Wilcoxon signed-rank test, introduced by Frank Wilcoxon in 1945, tests whether the median (more precisely, the pseudomedian) of paired differences is zero. It is the standard nonparametric alternative to the paired t-test when the distribution of within-pair differences is non-normal or when sample size is too small for the Central Limit Theorem to protect inference on the mean difference. The test operates on the paired differences themselves — not on the raw observations — and constructs its test statistic from the ranks of those differences' absolute values.

The mechanics proceed in four steps. First, compute the signed difference d_i = x_i2 - x_i1 for each pair i. Second, rank the absolute values |d_i| from smallest (rank 1) to largest (rank n), ignoring any pair with d_i = 0 by the standard Wilcoxon convention (more on this below). Third, reattach the sign of each original difference to its rank, yielding signed ranks. Fourth, compute W+ (sum of positive signed ranks) and W- (sum of negative signed ranks). The test statistic is conventionally reported as W = min(W+, W-). Under the null hypothesis that differences are symmetrically distributed around zero, W+ and W- should each be close to n(n+1)/4, and their total W+ + W- = n(n+1)/2 always holds as a useful arithmetic check.

The symmetry assumption — the one that often goes unacknowledged

The Wilcoxon signed-rank test is frequently described as "assumption-free" or as requiring only that the differences be independent. This is incorrect and the misunderstanding has real consequences in practice. The test requires that the distribution of differences be symmetric around its median, not merely that it be continuous. Under the null hypothesis the symmetry assumption is automatically satisfied (a distribution symmetric around zero satisfies any symmetry claim), but the test's power properties and interpretation under the alternative depend on symmetry holding. When the difference distribution is highly skewed — for example, most patients improve modestly and a few improve dramatically — the signed-rank test can reject for a reason other than a shift in location, because skewness itself disrupts the rank-sign balance the test exploits.

In RWE applications, cost changes (post - pre) are often right-skewed: most patients have moderate cost increases and a few have catastrophic spikes. Under this asymmetry, a significant Wilcoxon result does not cleanly identify a median shift — it may be detecting the skewness. The safe interpretation in this setting is that the two measurement occasions produce different rank patterns, not necessarily a symmetric shift in location.

Zeros and ties: the Pratt versus Wilcoxon method

In real data, some paired differences will be exactly zero (no change). The original Wilcoxon (1945) convention discards zero differences and reduces n accordingly. An alternative due to Pratt (1959) retains all pairs — zero differences are ranked but their contribution to W+ and W- cancels (they carry rank but no sign). In scipy.stats.wilcoxon, the `zero_method` argument controls this: `zero_method='wilcoxon'` (the default) drops zeros; `zero_method='pratt'` retains them. In R's wilcox.test(paired=TRUE), zeros are dropped by default. The choice matters when the fraction of zeros is non-trivial: dropping zeros reduces n and can inflate power if many zeros exist because of a true null; Pratt's method avoids this inflation. Document which method was used in the analysis SAP.

Ties in the absolute differences (two pairs with the same |d|) are handled by averaging the ranks they would occupy, exactly as in the Mann-Whitney U test. Large numbers of ties reduce the information content of the ranks; when ties are extensive (> 20% of pairs) the normal approximation for the p-value uses a tie-corrected variance. Exact p-values are available in small samples and are recommended when n < 25; for n ≥ 25 the normal approximation is adequate unless ties are heavy.

The pseudomedian and the Hodges-Lehmann estimator for the Wilcoxon signed-rank test

A key practical point that is frequently overlooked: the effect estimate that pairs naturally with the Wilcoxon signed-rank test is the pseudomedian of the differences — not the median of the differences, and not the difference of the medians. The pseudomedian of a sample is the median of all pairwise averages (d_i + d_j)/2 for i ≤ j, including d_i with itself (the Walsh averages). For a symmetric distribution the pseudomedian equals the median, but for a skewed difference distribution the two can diverge substantially. The Hodges-Lehmann estimator implements the pseudomedian and is the standard companion estimate for the Wilcoxon signed-rank test; in R, wilcox.test(paired=TRUE, conf.int=TRUE) returns it automatically. Reporting only a Wilcoxon p-value without this effect estimate leaves the reader unable to assess the magnitude or direction of the change.

The distinction matters operationally: if the analysis question is "did patients improve on average?" the mean difference (paired t-test or a mixed model) is the right target. If the question is "did a typical patient improve?" the median or pseudomedian of the differences is appropriate. In HEOR, cost-effectiveness models and budget-impact analyses need the mean change, not the pseudomedian, so the Wilcoxon signed-rank test is usually a sensitivity check alongside a model-based mean-change estimator for cost outcomes.

The sign test as the weaker, assumption-free fallback

When the symmetry assumption for the signed-rank test is clearly violated and the analyst wants a strictly assumption-free test for paired data, the sign test is the fallback. The sign test counts only whether each d_i is positive or negative (discarding magnitude entirely) and tests whether positive differences are equally likely as negative ones under the null. Because it discards rank information, the sign test is markedly less powerful than the Wilcoxon signed-rank test; it should be reserved for situations where the symmetry assumption is demonstrably implausible and the analyst cannot use a model-based approach.

RWE and HEOR applications

Pre-post healthcare utilization and costs: When the same patients are observed before and after a treatment initiation, procedure, or policy change and the outcome is an inherently skewed measure (emergency department visits, total allowed costs, inpatient days), the paired t-test on the within-patient difference can be distorted by outliers. The Wilcoxon signed-rank test provides a robust sensitivity check. However, as noted above, if the mean cost change is the decision-relevant quantity (for a budget-impact model), the Wilcoxon test is supplementary — a gamma GLM or bootstrap mean-difference estimator should be the primary analysis.

Patient-reported outcomes (PROs) and utility scores: EQ-5D utility scores, SF-6D scores, and disease-specific PROs often violate normality: they are bounded (0 to 1 or 0 to 100), may cluster near boundary values, and have pre-post differences that are neither continuous nor bell-shaped. The Wilcoxon signed-rank test is the standard nonparametric paired test in this context and is frequently required alongside the parametric test in health technology assessment (HTA) submissions.

Matched cohort designs: When 1:1 matched pairs are constructed in an observational study (propensity-score matched, exact-matched, or coarsened exact matched), the paired structure must be respected. A two-sample t-test or Mann-Whitney on matched pairs ignores the pairing and loses efficiency. The Wilcoxon signed-rank test correctly exploits the paired design by operating on within-pair differences, exactly as in the pre-post setting.

Pseudomedian-of-differences ≠ difference-of-medians — why this matters

In practice, analysts sometimes report the median change as "median(after) - median(before)" which is the difference of two group medians. This is not what the Wilcoxon signed-rank test tests, and it is not the pseudomedian of the differences. Consider a simple example: if half the patients improve from 4 to 8 (+4) and half improve from 5 to 9 (+4), both medians shift by exactly 4 and the pseudomedian of differences is 4. But if pre-scores are 4 and 5 (median 4.5) and post-scores are 8 and 9 (median 8.5), then the difference of medians is 4.0 and the pseudomedian is also 4.0 — they agree. However, in skewed real data they can diverge. The correct companion to the Wilcoxon test is always the Hodges-Lehmann pseudomedian; report it.

Pros, cons, and trade-offs

Pros: - Valid when the paired t-test's normality assumption is violated and n is small. - Robust to outliers in the differences: an extreme outlier contributes at most rank n rather than distorting the mean as it would in a paired t-test. - Exact p-values are available for small n (< 25), making it reliable in pilot or registry sub-studies with limited sample sizes. - The Hodges-Lehmann pseudomedian with confidence interval is a principled effect estimate that pairs naturally with the test. - Widely expected by regulators (FDA, EMA, NICE) as a sensitivity check alongside a parametric primary analysis.

Cons: - The symmetry assumption is often unacknowledged and can be violated in practice. - Lower power than the paired t-test when the normality assumption holds (the paired t-test uses the actual magnitude of differences, the signed-rank test uses only their ranks). - The pseudomedian is not the effect estimate that budget-impact or cost-effectiveness models require — they need the mean change. - Cannot be directly extended to covariate adjustment. Regression models on the within-pair differences (ANCOVA or a mixed model) are needed when baseline imbalance or confounders must be controlled. - Zero differences require a decision (Wilcoxon vs Pratt method) that should be pre-specified in the SAP.

Trade-offs vs alternatives: - Against the paired t-test: prefer the Wilcoxon signed-rank test when n < 30 and the difference distribution is clearly non-normal or when outliers are expected; prefer the paired t-test (or a mixed model) when the mean change is the target estimand. - Against the sign test: the Wilcoxon signed-rank test is almost always more powerful than the sign test; use the sign test only if the symmetry assumption is clearly untenable. - Against a regression/GLM on differences: when covariate adjustment is needed or the outcome is a count or cost, a regression approach (Poisson, gamma, or negative binomial with a paired-design term) is superior to either the Wilcoxon test or the paired t-test.

When to use

• Paired or pre-post continuous outcomes (or ordinal outcomes with many levels) where the
• Small paired samples (n < 30) where the Central Limit Theorem cannot be assumed to protect
• Matched-pair observational cohorts where within-pair differences are the natural estimand.
• Patient-reported outcomes and utility scores with non-normal or bounded difference
• As a pre-specified sensitivity analysis alongside a parametric primary test for any paired
• When the question is whether a typical patient improved (pseudomedian), not whether the

When NOT to use

• Independent groups: the Wilcoxon signed-rank test requires paired data; for two
• When the mean change is the estimand: if the result will feed a cost-effectiveness model,
• Count or zero-inflated pre-post outcomes: for outcomes like number of hospitalizations
• More than two timepoints: the Wilcoxon signed-rank test compares a single pair. For
• When covariate adjustment is needed: rank tests cannot incorporate baseline covariates
• When the symmetry assumption is demonstrably violated and the sample is small: if the

Interpreting the output

In the worked example, six pain patients were assessed before and after treatment. The sum of positive signed ranks is W+ = 15 and the sum of negative signed ranks is W− = 6; the check W+ + W− = 21 = 6 × 7 / 2 confirms the arithmetic. The test statistic is W = min(15, 6) = 6. The exact two-sided p-value at n = 6 is approximately 0.22. The Hodges-Lehmann pseudomedian of the within-patient differences is approximately 2.5 pain-score points of reduction.

(1) Formal interpretation. Under the null hypothesis that the paired differences are symmetrically distributed around zero, a W statistic of 6 or smaller in either tail has a probability of approximately 0.22 — well above alpha = 0.05. The test does not reject the null. At n = 6, the exact distribution of W has very limited resolution: only a small number of rank arrangements exist, so the test has low power to detect moderate effects. The pseudomedian of approximately 2.5 points indicates a positive directional trend, but the sample is far too small to confirm the signal statistically.

(2) Practical interpretation. All six patients showed some reduction in pain, and the Hodges-Lehmann pseudomedian of approximately 2.5 points is the appropriate effect estimate to report alongside this test — it is the pseudomedian of the within-patient differences, not the difference between the pre- and post-period medians. The non-significant p-value at n = 6 is better read as "inconclusive" than "no effect." The symmetry assumption required by the signed-rank test should be assessed; if the difference distribution is strongly skewed (most patients improve slightly, a few improve dramatically), a model-based analysis may be more appropriate, as the signed-rank test can detect skewness rather than a pure location shift.

Worked example

Scenario

A HEOR analyst is evaluating a 12-week pain management program in six patients. Each patient rates their pain on a 0-to-10 scale at baseline (week 0) and at week 12. The analyst wants to test whether pain scores changed significantly, but with only six patients the normality assumption for a paired t-test cannot be verified, so the Wilcoxon signed-rank test is chosen. All six differences have distinct absolute values and none equals zero, giving a clean illustration of the core mechanics.

Dataset

Pain scale ratings (0 = no pain, 10 = worst pain) for six patients before and after a 12-week pain management program. Difference = baseline score minus week-12 score; a positive difference means improvement (pain decreased).

Steps

• Step 1 — compute differences. Difference = baseline minus week-12: P1 = 7-2 = 5, P2 = 6-3 = 3, P3 = 4-6 = -2, P4 = 8-2 = 6, P5 = 5-4 = 1, P6 = 3-7 = -4. Positive values mean pain decreased (improvement); negative values mean pain increased (worsening). No difference equals zero, so no pairs need to be dropped.

• Step 2 — rank the absolute differences from smallest to largest. |P5|=1 gets rank 1, |P3|=2 gets rank 2, |P2|=3 gets rank 3, |P6|=4 gets rank 4, |P1|=5 gets rank 5, |P4|=6 gets rank 6. All absolute values are distinct so no averaging of tied ranks is needed.

• Step 3 — reattach signs. P1: +5, P2: +3, P3: -2, P4: +6, P5: +1, P6: -4. Positive signed ranks go to W+; negative signed ranks (by absolute value) go to W-.

• Step 4 — compute W+ and W-. W+ = ranks of patients who improved = 5 + 3 + 6 + 1 = 15. W- = ranks of patients who worsened = 2 + 4 = 6. Verify: W+ + W- = 15 + 6 = 21 = 6*7/2 = 21. The check passes.

• Step 5 — the test statistic is W = min(W+, W-) = min(15, 6) = 6. A small W means most of the large ranks cluster on one side (here, the positive/improvement side), which is evidence against the null hypothesis of no change.

• The exact two-sided p-value for W = 6 with n = 6 is 0.219, which does not reach the conventional alpha = 0.05 threshold. With only six patients, the test has low power; the four-to-two split in improvement direction is suggestive but not statistically significant at this sample size.

Result

W+ = 5 + 3 + 6 + 1 = 15, W- = 2 + 4 = 6, W+ + W- = 15 + 6 = 21 = 6*7/2 = 21 (check passes). Test statistic W = min(15, 6) = 6. Four of six patients improved (positive differences) and those improvements carried the larger ranks (5, 3, 6, 1), while the two worsening patients had smaller absolute ranks (2, 4). The pseudomedian of the six differences is the median of all 21 Walsh averages; the Hodges-Lehmann estimate equals 2.5 points of pain reduction. Exact two-sided p ≈ 0.22 — not significant at alpha = 0.05 with n = 6, illustrating that small samples require larger effect sizes to achieve significance.

Runnable example

from scipy import stats
import itertools

# ── Data: six paired pain scores (baseline minus week-12, positive = improvement) ──
baseline = [7, 6, 4, 8, 5, 3]
week12   = [2, 3, 6, 2, 4, 7]
diffs = [b - a for b, a in zip(baseline, week12)]
# diffs = [5, 3, -2, 6, 1, -4]  -> 4 positive, 2 negative

# ── 1. Wilcoxon signed-rank test (default: zero_method='wilcoxon', exact p-value at small n) ──
# Note: scipy.stats.wilcoxon takes the differences directly (or x and y separately).
w_stat, p_val = stats.wilcoxon(diffs, zero_method='wilcoxon', mode='auto')
print(f"Wilcoxon signed-rank test:")
print(f"  W statistic (sum of positive ranks) = {w_stat:.1f}")
print(f"  Two-sided p-value = {p_val:.4f}")
print(f"  (With n=6 scipy uses exact enumeration; normal approximation used for n >= 25)")

# ── 2. Verify W+ and W- by hand ──
abs_diffs = [abs(d) for d in diffs]
# Sort indices by absolute value to assign ranks
sorted_idx = sorted(range(len(abs_diffs)), key=lambda i: abs_diffs[i])
ranks = [0] * len(diffs)
for rank_pos, orig_idx in enumerate(sorted_idx):
    ranks[orig_idx] = rank_pos + 1   # ranks are 1-indexed
W_plus  = sum(r for d, r in zip(diffs, ranks) if d > 0)
W_minus = sum(r for d, r in zip(diffs, ranks) if d < 0)
n = len(diffs)
print(f"\nManual verification:")
print(f"  W+ (sum of positive ranks) = {W_plus}")
print(f"  W- (sum of negative ranks) = {W_minus}")
print(f"  W+ + W- = {W_plus + W_minus}  (should equal n*(n+1)/2 = {n*(n+1)//2})")
print(f"  W = min(W+, W-) = {min(W_plus, W_minus)}")

# ── 3. Pratt method (retains zero differences, assigns ranks but no sign contribution) ──
# Not relevant here (no zeros), but shown for completeness
diffs_with_zero = diffs + [0]   # artificial example with a zero
w_pratt, p_pratt = stats.wilcoxon(diffs_with_zero, zero_method='pratt', mode='approx')
print(f"\nPratt method example (n=7 including one zero):")
print(f"  W = {w_pratt:.1f}, p (normal approx) = {p_pratt:.4f}")

# ── 4. Hodges-Lehmann pseudomedian: median of all Walsh averages (d_i + d_j)/2 for i <= j ──
# This is the correct companion effect estimate for the Wilcoxon signed-rank test.
n = len(diffs)
walsh = [(diffs[i] + diffs[j]) / 2 for i in range(n) for j in range(i, n)]
walsh.sort()
nw = len(walsh)
if nw % 2 == 1:
    pseudomedian = walsh[nw // 2]
else:
    pseudomedian = (walsh[nw // 2 - 1] + walsh[nw // 2]) / 2
print(f"\nHodges-Lehmann pseudomedian of differences: {pseudomedian:.2f}")
print(f"  (Number of Walsh averages = n*(n+1)/2 = {n*(n+1)//2}, computed {nw})")
print(f"  Pseudomedian ≠ simple median of differences ({sorted(diffs)[nw//2 - 1]:.1f})")
print("  Always report this estimate with a CI — p-value alone does not convey effect size.")

# ── 5. Normal approximation for large n ──
import math
n_large = 50  # example large sample
w_example = 450  # hypothetical W+ statistic
mu_w = n_large * (n_large + 1) / 4
sigma_w = math.sqrt(n_large * (n_large + 1) * (2 * n_large + 1) / 24)
z = (w_example - mu_w - 0.5) / sigma_w   # continuity correction of 0.5
from scipy.stats import norm
p_approx = 2 * norm.sf(abs(z))
print(f"\nNormal approximation (n=50, W+={w_example}):")
print(f"  mu_W = {mu_w:.1f}, sigma_W = {sigma_w:.3f}")
print(f"  z = ({w_example} - {mu_w:.1f} - 0.5) / {sigma_w:.3f} = {z:.3f}")
print(f"  Two-sided p (normal approx) = {p_approx:.4f}")

# ── Data: six paired pain scores ──
baseline <- c(7, 6, 4, 8, 5, 3)
week12   <- c(2, 3, 6, 2, 4, 7)
diffs    <- baseline - week12   # c(5, 3, -2, 6, 1, -4)

# ── 1. Wilcoxon signed-rank test with Hodges-Lehmann pseudomedian and CI ──
# conf.int=TRUE returns the Hodges-Lehmann estimate and 95% CI automatically.
# R drops zero differences by default (Wilcoxon convention).
# exact=TRUE uses exact enumeration (recommended at n < 25).
wsr_res <- wilcox.test(baseline, week12, paired = TRUE,
                       conf.int = TRUE, exact = TRUE)
cat("Wilcoxon signed-rank test:\n")
print(wsr_res)
# The output includes:
#   V: the W+ statistic (sum of positive signed ranks); R calls it V
#   p-value: exact two-sided p-value
#   pseudomedian: Hodges-Lehmann estimate
#   conf.int: 95% CI for the pseudomedian
cat(sprintf("\nHodges-Lehmann pseudomedian = %.2f\n", wsr_res$estimate))
cat(sprintf("95%% CI: [%.2f, %.2f]\n", wsr_res$conf.int[1], wsr_res$conf.int[2]))
cat("Note: R labels W+ as 'V'; scipy labels it 'statistic'. Both equal sum of positive ranks.\n")

# ── 2. Manual W+ / W- calculation to match worked example ──
abs_d <- abs(diffs)
ranks <- rank(abs_d)   # R's rank() handles ties by averaging
W_plus  <- sum(ranks[diffs > 0])
W_minus <- sum(ranks[diffs < 0])
n <- length(diffs)
cat(sprintf("\nManual verification:\n  W+ = %.0f,  W- = %.0f\n", W_plus, W_minus))
cat(sprintf("  W+ + W- = %.0f  (should be n*(n+1)/2 = %d)\n", W_plus + W_minus, n*(n+1)/2))
cat(sprintf("  W = min(W+, W-) = %.0f\n", min(W_plus, W_minus)))

# ── 3. Normal approximation for large n (exact=FALSE) ──
# Demonstrate on a larger hypothetical dataset
set.seed(42)
large_pre  <- round(runif(60, 4, 9))
large_post <- round(runif(60, 2, 8))
wsr_large <- wilcox.test(large_pre, large_post, paired = TRUE,
                         conf.int = TRUE, exact = FALSE)
cat("\nLarge sample example (n=60, normal approximation):\n")
print(wsr_large)

# ── 4. Paired t-test for comparison ──
# Show that both tests agree in direction but may differ in p-value
t_res <- t.test(baseline, week12, paired = TRUE)
cat(sprintf("\nPaired t-test for comparison:\n  t = %.3f,  p = %.4f\n",
            t_res$statistic, t_res$p.value))
cat(sprintf("  Mean difference = %.2f (95%% CI: [%.2f, %.2f])\n",
            t_res$estimate, t_res$conf.int[1], t_res$conf.int[2]))
cat("Note: At n=6, both tests are underpowered; with this data neither reaches p < 0.05.\n")