Mann-Whitney U Test (Wilcoxon Rank-Sum)

What the Mann-Whitney U test actually tests

The Mann-Whitney U test (also called the Wilcoxon rank-sum test) is a nonparametric two-sample test that operates on ranks rather than raw values. Both names refer to the identical procedure: Mann and Whitney (1947) derived the U statistic via pairwise comparisons, while Wilcoxon (1945) derived the rank-sum form; they are algebraically identical. The test answers one specific question: is there stochastic dominance between the two groups?

Under the null hypothesis, the probability that a randomly drawn observation from group 1 exceeds a randomly drawn observation from group 2 equals 0.5. Formally, H0: P(X > Y) = 0.5 for independent X ~ F and Y ~ G.

A pervasive misconception — worth stating explicitly — is that the Mann-Whitney test is a "test of medians." Divine et al. (2018) demonstrated rigorously that this folklore is incorrect. The Mann-Whitney null of equal medians holds ONLY under the additional assumption that the two distributions have identical shapes differing only by a location shift. In practice, if the two groups differ in variance, skewness, or tail behavior while having equal medians, the Mann-Whitney test can reject.

Conversely, two groups with different medians can fail to reject if the distributions differ in ways that cancel out in the rank ordering. When reporting a Mann-Whitney result, the correct language is "group 1 tended to have higher [outcome]" or "there is evidence of stochastic dominance," not "the medians differ."

Computing U: the mechanics

Pool all observations from both groups and rank them from smallest (rank 1) to largest (rank N, where N = n1 + n2). When two or more values tie, each tied value receives the average of the ranks they would have occupied. Let R1 = sum of ranks for group 1. The U statistic for group 1 is:

U1 = R1 - n1(n1 + 1) / 2

The complementary U for group 2 is U2 = n1·n2 - U1. As a check, U1 + U2 = n1·n2 always, and the total rank sum R1 + R2 = N(N+1)/2. The conventional test statistic is U = min(U1, U2), and the p-value is obtained from the exact distribution (for small samples) or the normal approximation (for large samples or with many ties).

The U-to-AUC connection: a genuinely useful HEOR interpretation

The ratio U1 / (n1·n2) equals the empirical probability index — the fraction of all n1·n2 pairwise comparisons in which a group-1 observation exceeds a group-2 observation. This is numerically identical to the area under the receiver operating characteristic (ROC) curve (AUC) when group membership is the binary label. An AUC of 0.50 means no discrimination; an AUC of 0.80 means a randomly chosen group-1 patient has an 80% probability of a larger outcome than a randomly chosen group-2 patient.

This restatement converts an abstract rank statistic into an immediately communicable effect size that health technology assessment bodies understand — especially for patient-reported outcomes, quality-of-life scores, and functional assessments.

The Hodges-Lehmann shift estimate: the effect-size companion

A bare p-value from a Mann-Whitney test is insufficient for HEOR decision-making. The Hodges-Lehmann (HL) estimator provides the natural location estimate to accompany the test. The HL estimate is the median of all n1·n2 pairwise differences (xi - yj), and its associated confidence interval uses the Wilcoxon distribution. In R, `wilcox.test(..., conf.int = TRUE)` returns the HL estimate and CI directly.

In SAS, `PROC NPAR1WAY WILCOXON HL` produces both. The HL estimate is neither the difference in means nor the difference in medians — it is the "shift" that best explains the rank ordering, and it is on the original outcome scale (e.g., days of hospitalization).

Report it: "Group A had a median pairwise shift of -1.5 days [95% CI -3.0 to -0.5 days] fewer hospitalization days than group B." This gives the payer or clinical decision-maker a number to act on, not just a rejection flag.

Ties and their consequences

Ties arise whenever two or more observations share the same value. In claims data this is common — length of stay (LOS) is measured in integer days, so a dataset with 5,000 patients might have 800 patients at LOS = 1, 600 at LOS = 2, etc. Ties are handled by midrank averaging (each tied observation receives the mean of the ranks it would have occupied). With many ties the test statistic's variance is underestimated under the standard formula, requiring a continuity correction.

All major software packages (scipy.stats.mannwhitneyu with `method='asymptotic'`, R `wilcox.test` with `exact=FALSE`, SAS `PROC NPAR1WAY WILCOXON`) apply the tie correction automatically. When ties are extremely heavy — e.g., an ordinal outcome with only five distinct values across 10,000 observations — the Mann-Whitney test loses power and a proportional-odds logistic regression becomes a more informative analysis.

RWE realities: costs, LOS, and the routing decision

Length of stay and total healthcare costs are the canonical skewed outcomes in RWE and HEOR. The Mann-Whitney test is a valid choice for comparing the rank ordering of these outcomes between treatment groups, but it answers a different question from what most budget-impact analyses and payer submissions require. Payers need a mean cost difference — the quantity that multiplies by population size to yield total budget impact. The Mann-Whitney U test does NOT estimate mean costs, and the HL shift estimate is NOT a mean difference.

For mean-cost inference, the modern standard is a generalized linear model (GLM) with a log link and a gamma or Tweedie variance function, which

1. targets the mean on the original dollar scale,
2. handles the right-skewed distributional shape, and
3. accommodates covariate adjustment. The Mann-Whitney is appropriate as a descriptive or sensitivity check — "does group A tend to have lower LOS?" — not as the primary analysis when mean costs are the policy-relevant estimand.

Adjustment and causality: what rank tests cannot do

The Mann-Whitney test is a bivariate comparison. In an unbalanced observational dataset, an unadjusted Mann-Whitney comparing treated versus untreated patients estimates a potentially confounded association, not a causal treatment effect. There is no natural regression extension of the Mann-Whitney test that adjusts for covariates the way a GLM, propensity-weighted model, or g-formula can. When confounding is a concern — which it always is in non-randomized RWE — route to propensity-score weighting or matching followed by a weighted t-test or GLM, not to a bare Mann-Whitney.

The test is most defensible for primary analysis in randomized data or as a sensitivity check in a pre-specified non-inferiority analysis after achieving covariate balance by design.

Pros, cons, and trade-offs

Pros:

• Valid without distributional assumptions (no normality requirement); robust to heavy tails and outliers, which dampens the influence of extreme cost or LOS values to a single high rank.
• Exact p-values available for small samples via the permutation distribution of ranks.
• The U/(n1·n2) probability index and ROC AUC equivalence provide an intuitive, directly communicable effect-size measure.
• Widely implemented and understood; acceptable to FDA, EMA, and HTA bodies as a pre-specified nonparametric sensitivity analysis.
• Well-suited for ordinal outcomes where values are intrinsically ranked but not continuous.

Cons:

• Does not test medians (unless equal-shape assumption holds); this is routinely misreported and must be explicitly corrected in manuscripts and regulatory submissions.
• The HL shift estimate is not a mean difference; it is a poor substitute for the mean-cost difference that payers need for budget-impact models.
• No covariate-adjustment framework comparable to regression; cannot control confounding directly.
• Power is lower than a t-test when the parametric normality assumption holds; in large RWE datasets this efficiency loss is negligible but in small studies it may matter.
• Heavy ties (common in integer-valued claims variables) reduce power and require software tie-correction; with very few unique values a proportional-odds model is preferable.

When NOT to use

• When the primary decision requires a mean cost difference: the HL shift is not interpretable as a mean-cost difference; route primary inference to a gamma GLM or two-part model.
• When confounding is uncontrolled: an unadjusted Mann-Whitney on an unbalanced observational dataset estimates a confounded association, not a causal effect. Use propensity-weighted GLM or matching methods instead.
• For paired data (pre-post measurements or matched pairs): use the Wilcoxon signed-rank test (the paired analogue) rather than the rank-sum test, which assumes independence between groups.
• When the outcome is ordinal with very few levels (e.g., a 5-point Likert scale with most responses at 3-4): proportional-odds logistic regression or a cumulative link model captures the full ordinal structure and allows covariate adjustment; Mann-Whitney reduces this to a rank ordering that discards distributional detail.
• When the goal is an estimate of relative risk, hazard ratio, or odds ratio for a binary outcome: use logistic regression, Poisson GLM with robust SE, or a log-binomial model.
• When reporting, do NOT describe a Mann-Whitney result as a test of medians unless you explicitly state and verify the equal-shape assumption — this is one of the most common statistical errors in HEOR manuscripts.

Interpreting the output

In the worked example, the rank sums for Group A (shorter hospital stays, n = 4) and Group B (longer stays, n = 4) are R_A = 12 and R_B = 24. The U statistic for Group A is U_A = 2 and for Group B is U_B = 14; U_A + U_B = 16 = 4 × 4, confirming the computation.

The test statistic is U = min(2, 14) = 2. The probability index — U_B / (n_A × n_B) = 14/16 = 0.875 — is the fraction of all 16 pairwise Group A vs Group B comparisons in which the Group B observation is larger.

(1) Formal interpretation. Under the null of stochastic equality (P(X_A > X_B) = 0.5), the expected value of U for either group is n_A × n_B / 2 = 8. The observed U_A = 2 is far below this expectation. The probability index of 0.875 indicates that Group B stochastically dominates Group A — a randomly chosen Group B patient has a larger observation than a randomly chosen Group A patient in 87.5% of pairings.

The exact p-value (available from the permutation distribution at this small n) should be compared to the pre-specified alpha. The test evaluates stochastic dominance — whether P(X > Y) = 0.5 — not whether the group medians are equal; median equality is only entailed under the additional assumption that both distributions have the same shape, differing only in location.

(2) Practical interpretation. Group A tends toward shorter hospital stays: a randomly selected Group A patient had a shorter stay than a randomly selected Group B patient in 87.5% of all possible pairings. This probability index is directly analogous to the ROC AUC and is immediately communicable as a measure of stochastic dominance.

For budget-impact or resource- planning contexts where mean length of stay is the required input, a Hodges-Lehmann shift estimate (the median of all pairwise differences between groups) or a regression-based mean estimate should accompany this rank-based result — the U statistic alone is not a mean difference and cannot be plugged directly into a cost model.

Decision diagram

flowchart TD
  Q[Two independent groups,<br/>continuous or ordinal outcome?] --> Size{Sample size<br/>per group}
  Size -->|n < 50, non-normal<br/>or ordinal| MW["Mann-Whitney U Test<br/>(rank-sum)<br/>H0: P(X>Y) = 0.5"]
  Size -->|n ≥ 50 or<br/>approximately normal| T["Welch t-test<br/>(parametric default)<br/>tests mean difference"]
  Size -->|skewed cost/LOS,<br/>need mean difference| GLM["Gamma GLM with log link<br/>(primary for mean inference)"]
  MW --> Effect["Report effect size:<br/>U/(n1·n2) probability index<br/>+ Hodges-Lehmann shift CI"]
  T --> EffectT["Report mean difference<br/>with 95% CI"]
  GLM --> EffectGLM["Report mean ratio or<br/>mean difference with 95% CI"]
  MW -.->|paired/matched data?| SR["Use Wilcoxon<br/>signed-rank test instead"]
  MW -.->|3+ groups?| KW["Use Kruskal-Wallis<br/>+ Dunn post-hoc instead"]

Worked example

Scenario

A health outcomes analyst is comparing length of stay (LOS) in days for two groups of patients hospitalized with a respiratory infection: 4 patients receiving a new antibiotic regimen (Group A) and 4 patients receiving standard of care (Group B). The analyst wants to test whether Group A tends to have shorter stays using the Mann-Whitney U test, and then compute the U statistic and the probability index (U / n1*n2) to see how strongly Group A dominates Group B in the rank ordering. The dataset has no tied values so the ranking is straightforward.

Dataset

Length of stay in days (n=4 per group). No tied values. All 8 observations are pooled and ranked together for the Mann-Whitney test.

Steps

Result

R_A = 12, R_B = 24, total = 36 = 8*9/2. U_A = 12 - 10 = 2, U_B = 24 - 10 = 14, U_A + U_B = 2 + 14 = 16 = 4*4. Probability index = 14/16 = 0.875. Group A patients tend to have shorter hospital stays than Group B patients; the new antibiotic arm dominated in 87.5% of all pairwise comparisons. Note: this does not test medians — it tests stochastic dominance. Pair this with a Hodges-Lehmann shift estimate and CI for a reportable effect size.

Trade-offs

Runnable example

from scipy import stats
import statistics

# ── Motivating dataset: LOS in days, n=4 per group, no ties ──
group_a = [2, 3, 5, 6]   # new antibiotic: shorter stays
group_b = [4, 7, 8, 9]   # standard care: longer stays

# ── 1. Mann-Whitney U test ──
# alternative='two-sided'  tests H0: P(X>Y) = 0.5 (stochastic dominance), NOT medians
# method='asymptotic'      uses normal approximation with tie correction (valid at large n or with ties)
# method='exact'           uses permutation distribution (no ties; small n only)
u_stat, p_val = stats.mannwhitneyu(group_a, group_b, alternative="two-sided", method="asymptotic")
n1, n2 = len(group_a), len(group_b)
print(f"U statistic (group A): {u_stat:.1f}")
print(f"p-value (two-sided):   {p_val:.4f}")
print("Null hypothesis: P(X_A > X_B) = 0.5  [stochastic dominance, NOT a test of medians]")

# ── 2. Probability index = AUC equivalent ──
# U / (n1 * n2) = empirical P(A > B) = ROC AUC when group = binary label
prob_index = u_stat / (n1 * n2)
print(f"\nProbability index (U / n1*n2): {prob_index:.3f}")
print(f"Interpretation: in {prob_index*100:.1f}% of pairwise comparisons a Group A patient")
print(f"  had a shorter stay than a Group B patient. (AUC = {prob_index:.3f})")

# ── 3. Hodges-Lehmann location estimate ──
# Median of all n1*n2 pairwise differences (a - b for a in A, b in B)
pairwise_diffs = sorted(a - b for a in group_a for b in group_b)
n = len(pairwise_diffs)
hl_estimate = (pairwise_diffs[n // 2 - 1] + pairwise_diffs[n // 2]) / 2 if n % 2 == 0 \
              else pairwise_diffs[n // 2]
print(f"\nHodges-Lehmann shift estimate (A - B): {hl_estimate:.1f} days")
print("This is NOT the mean difference nor the median difference.")
print("Report as: 'Group A had a pairwise median shift of X days [95% CI via R/SAS]'")

# ── 4. Manual rank-sum verification ──
all_vals = sorted(enumerate(group_a + group_b), key=lambda x: x[1])
ranks = {}
i = 0
while i < len(all_vals):
    j = i
    while j < len(all_vals) - 1 and all_vals[j][1] == all_vals[j + 1][1]:
        j += 1
    midrank = (i + 1 + j + 1) / 2   # average rank (1-indexed)
    for k in range(i, j + 1):
        ranks[all_vals[k][0]] = midrank
    i = j + 1
r_a = sum(ranks[i] for i in range(n1))
r_b = sum(ranks[i] for i in range(n1, n1 + n2))
u_a = r_a - n1 * (n1 + 1) / 2
u_b = r_b - n2 * (n2 + 1) / 2
print(f"\nManual check: R_A={r_a:.1f}, R_B={r_b:.1f}, total={r_a+r_b:.1f} (expect {(n1+n2)*(n1+n2+1)/2:.1f})")
print(f"U_A={u_a:.1f}, U_B={u_b:.1f}, U_A+U_B={u_a+u_b:.1f} (expect {n1*n2})")

# ── Motivating dataset: LOS in days, n=4 per group ──
group_a <- c(2, 3, 5, 6)   # new antibiotic
group_b <- c(4, 7, 8, 9)   # standard care

# ── 1. Mann-Whitney U test (two-sided; R calls it Wilcoxon rank-sum) ──
# exact = FALSE: use normal approximation (required when ties present or n is large)
# exact = TRUE:  exact permutation p-value (small n, no ties only)
mw_res <- wilcox.test(group_a, group_b, alternative = "two-sided", exact = FALSE)
cat("Mann-Whitney / Wilcoxon rank-sum test:\n")
print(mw_res)
cat("Note: this tests P(A > B) = 0.5 (stochastic dominance), NOT equality of medians.\n\n")

# ── 2. Hodges-Lehmann location estimate + 95% CI ──
# conf.int = TRUE activates the HL estimate alongside the test
hl_res <- wilcox.test(group_a, group_b, alternative = "two-sided",
                      exact = FALSE, conf.int = TRUE)
cat(sprintf("Hodges-Lehmann shift estimate (A - B): %.1f days\n", hl_res$estimate))
cat(sprintf("95%% CI: [%.1f, %.1f] days\n\n", hl_res$conf.int[1], hl_res$conf.int[2]))
cat("Report: 'Group A had a pairwise median shift of X days [95% CI ...] fewer than Group B'\n")

# ── 3. Probability index (AUC equivalent) ──
n1 <- length(group_a)
n2 <- length(group_b)
# W in wilcox.test is the U statistic for group_a vs group_b
u_a <- mw_res$statistic
prob_index <- u_a / (n1 * n2)
cat(sprintf("Probability index (U / n1*n2) = %.1f / %d = %.3f\n", u_a, n1 * n2, prob_index))
cat(sprintf("Group A exceeds Group B in %.1f%% of pairwise comparisons (AUC = %.3f)\n\n",
            prob_index * 100, prob_index))

# ── 4. Manual rank-sum verification ──
all_vals <- c(group_a, group_b)
group_label <- c(rep("A", n1), rep("B", n2))
rks <- rank(all_vals)   # ties.method = "average" by default (midrank)
r_a <- sum(rks[group_label == "A"])
r_b <- sum(rks[group_label == "B"])
u_a_manual <- r_a - n1 * (n1 + 1) / 2
u_b_manual <- r_b - n2 * (n2 + 1) / 2
cat(sprintf("Manual: R_A=%.1f, R_B=%.1f, total=%.1f (expect %d)\n",
            r_a, r_b, r_a + r_b, (n1 + n2) * (n1 + n2 + 1) / 2))
cat(sprintf("U_A=%.1f, U_B=%.1f, sum=%.1f (expect %d)\n",
            u_a_manual, u_b_manual, u_a_manual + u_b_manual, n1 * n2))

/* ── Create motivating dataset: LOS in days, n=4 per group ── */
data work.los;
  input patient_id $ group $ los_days;
  datalines;
A1 A 2
A2 A 3
A3 A 5
A4 A 6
B1 B 4
B2 B 7
B3 B 8
B4 B 9
;
run;

/* ── Mann-Whitney / Wilcoxon rank-sum test with Hodges-Lehmann ──
   WILCOXON: Wilcoxon rank-sum statistic = Mann-Whitney U (same test)
   HL:       Hodges-Lehmann location estimate and 95% CI
   The default p-value uses the normal approximation (appropriate for large n or ties).
   Add the EXACT WILCOXON statement (see below) for exact permutation p-value at small n.
── */
proc npar1way data=work.los wilcoxon hl;
  class group;       /* two-level classification variable: A vs B */
  var los_days;
  /* WILCOXON option produces:
       - Wilcoxon Two-Sample Test: Z statistic + asymptotic p-value
       - Kruskal-Wallis One-Way Analysis (identical for two groups)
     HL option produces:
       - Hodges-Lehmann point estimate of location shift (A - B)
       - 95% CI for the shift                                         */
run;

/* ── To obtain the exact permutation p-value (small n, no ties) ──
   Note: can be slow for large datasets; use MAXTIME= to limit runtime */
/*
proc npar1way data=work.los wilcoxon hl;
  class group;
  var los_days;
  exact wilcoxon / maxtime=60;
run;
*/

/* ── Probability index (U / n1*n2) from PROC NPAR1WAY output ──
   The Wilcoxon statistic W reported by SAS equals R_1 (rank sum for the first group
   alphabetically). Derive U and the probability index manually if needed:
     U_A = W - n_A * (n_A + 1) / 2
     Probability index = U_A / (n_A * n_B)
   For the motivating dataset: W = R_A = 12, n_A = 4, n_B = 4
     U_A = 12 - 4*5/2 = 12 - 10 = 2
     Probability index (Group A > Group B) = 2 / 16 = 0.125
     => Group B has the higher probability index: 14/16 = 0.875            */

Citations

1. [1]Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics. 1947;18(1):50-60.
2. [2]Divine GW, Norton HJ, Barón AE, Juarez-Colunga E. The Wilcoxon-Mann-Whitney procedure fails as a test of medians. The American Statistician. 2018;72(3):278-286.

1. [3]Hodges JL, Lehmann EL. Estimates of location based on rank tests. Annals of Mathematical Statistics. 1963;34(2):598-611.