Patient Preference Study (DCE / BWS)

A patient preference study measures stated preferences: how much respondents value the attributes of a treatment, service, or policy by asking them to choose among carefully constructed hypothetical alternatives. The two dominant elicitation formats are the discrete-choice experiment (DCE) — respondents repeatedly pick the preferred profile from sets of multi-attribute alternatives — and best-worst scaling (BWS) — respondents pick the best and worst item within a set (Case 1 = objects, Case 2 = attribute levels within a profile, Case 3 = full profiles).

Both rest on Lancaster's characteristics theory of value (utility derives from attributes, not the good itself) and McFadden's random utility model (RUM): the analyst recovers the parameters of a latent utility function from observed choices. The output is a vector of attribute-level utilities (part-worths) that, when one attribute is cost or a quantitative health outcome, yields willingness-to-pay (WTP) or maximum acceptable risk (MAR) trade-off metrics on a natural scale.

Core conceptual distinction

A preference study estimates a preference parameter (a marginal rate of substitution between attributes), not an epidemiologic parameter (incidence, hazard, risk difference) and not a clinical outcome. The estimand is the population (or latent-class, or individual-level distribution of) part-worth utilities — equivalently, the trade-off weights in respondents' utility function — typically reported as conditional-logit coefficients, marginal WTP, or attribute importance shares.

Three design choices define the method and are separable:

1. what is varied — the attributes and levels, chosen to be salient, non-overlapping, and policy-relevant;
2. how profiles are combined into choice tasks — the experimental design (full factorial is almost never feasible; a D-efficient fractional design minimizes the determinant of the variance-covariance matrix to maximize statistical efficiency for a fixed number of tasks); and
3. the choice rule modeled — RUM under IIA (conditional/multinomial logit), relaxed for preference heterogeneity (mixed/random-parameters logit, latent-class logit) or for scale heterogeneity (generalized multinomial logit, G-MNL).

DCE recovers a full utility surface and supports WTP; BWS Case 1 ranks discrete objects and is cognitively lighter but does not yield WTP unless cost is built in.

Pros, cons, and trade-offs

• vs revealed-preference / observational utilization analysis (claims, EHR): A DCE can value attributes that do not yet exist in the market (a pipeline drug's novel mechanism, a hypothetical mode of administration) and cleanly isolates the marginal value of a single attribute, which revealed choices confound with availability, price, formulary, and access. Cost: stated choices are hypothetical and subject to hypothetical bias — what people say they would do can diverge from what they do under real budget constraints and consequences. Prefer a DCE when the attribute or product is not yet observable in real-world data, or when you need a clean trade-off (e.g., regulatory benefit-risk).
• vs standard-gamble / time-trade-off utility elicitation (for QALYs): TTO/SG anchor health states on the 0–1 (dead–full health) QALY scale and feed cost-utility analysis directly; a DCE values attributes of care or treatment process, not just health states, and is more flexible but does not natively produce anchored utilities unless a duration/death attribute and appropriate anchoring are designed in (DCE-TTO hybrids). Prefer TTO/SG when the deliverable is a QALY weight; prefer a DCE when process, risk, and non-health attributes matter.
• vs qualitative interviews / focus groups: Qualitative work is essential upstream to identify and word attributes, but it cannot quantify trade-offs or produce WTP. A DCE quantifies; it cannot tell you why. Use them in sequence, not as substitutes — qualitative attribute development then quantitative DCE.
• vs simple rating/ranking or Likert importance scales: DCE/BWS force trade-offs and so avoid the lack of discrimination, yea-saying, and scale-use bias that plague direct importance ratings. Cost: higher respondent burden and design complexity.
• DCE vs BWS: BWS (especially Case 2) reduces cognitive load, avoids an explicit cost attribute, and is robust when respondents struggle with full profiles; it sacrifices the ability to compute WTP (Case 1/2) and the realism of choosing between whole products.

When NOT to use — and when it is actively misleading or dangerous

• When you need the actual rate of a behavior or a clinical effect. A DCE gives preference weights, never an incidence, adherence rate, or treatment effect. Reporting "60% would choose drug A" from a DCE as a real-world uptake forecast is a category error — predicted choice shares from a DCE are conditional on the experimental attribute set and ignore real access, price negotiation, and physician gatekeeping.
• When the attribute list is wrong or incomplete. If a dominant driver of real choice is omitted (omitted-attribute bias), or attributes overlap/correlate so respondents cannot trade them independently, the estimated part-worths are biased and the WTP is not interpretable. Skipping qualitative attribute development is the most common fatal error.
• Severe hypothetical bias / non-consequential survey. When respondents have no stake and the survey is purely hypothetical, magnitudes (especially WTP) can be inflated by multiples; without cheap-talk scripts, consequentiality framing, or external calibration the absolute numbers should not be taken at face value.
• Dominated designs and inattentive panels. If one alternative dominates on every attribute, or respondents straight-line / random-click, the choices carry no preference information; failure to include attention/dominance checks and to model scale heterogeneity yields confidently wrong estimates.
• Cognitively overloaded designs. Too many attributes/levels (>7 attributes, dozens of tasks) push respondents into simplifying heuristics (lexicographic, attribute non-attendance), violating the compensatory RUM the analysis assumes.
• Generalizing a convenience-panel sample to a clinical population. Online opt-in panels over-represent the healthy, literate, and internet-engaged; a preference estimate from such a frame may not transport to the sicker target population that actually faces the decision.

Data-source operational depth

Unlike pharmacoepidemiologic designs, a preference study generates its own primary data; the "data source" is the survey instrument, the sampling frame, and the elicited choice matrix, so the failure modes are measurement and sampling failures rather than claims/EHR artifacts.

• Online opt-in / access panels (the modal source): Fast and cheap but prone to sample-frame selection (panelists are healthier, younger, more literate, and financially motivated), professional-respondent and bot contamination, and straight-lining / speeding. Workarounds: recruit through patient organizations or clinics to reach the true target population, screen on a verified diagnosis, embed a dominated-pair (rationality) check and an instructional manipulation check, drop respondents below a completion-time threshold, and analyze scale heterogeneity (G-MNL) rather than assuming a homogeneous error variance.
• Clinic / point-of-care recruitment: Best for reaching genuinely affected patients and confirming diagnosis/severity from the chart, but slow, costly, and subject to consent-driven selection (sicker or more engaged patients enroll). Pre-register the sampling protocol and report the recruitment funnel.
• Pen-and-paper / interviewer-administered: Reduces drop-out and improves comprehension in low-literacy or elderly populations, but introduces interviewer effects and limits design complexity (no adaptive choice sets). Use simpler BWS or a small blocked DCE.
• Linked stated- and revealed-preference data: The strongest substrate for validating absolute magnitudes — calibrate hypothetical WTP against an observed market transaction or an incentive-compatible task — but linkage is rare, raises consent and privacy issues, and the revealed-preference benchmark carries its own confounding (price/access). Use it for calibration, not as the primary frame.
• Cross-cutting design failures regardless of mode: attribute-level imbalance or correlation that makes a parameter inestimable; insufficient choice-task count or sample size (use the de Bekker-de Vrieze / Orme rule of thumb, n ≥ 500·c / (t·a), with c = max levels of any attribute, t = tasks per respondent, a = alternatives per task, as a floor, not a power calculation); ordering/learning effects (randomize task order and block the design); and ignoring panel structure (multiple tasks per respondent are correlated — cluster standard errors or use a panel mixed logit).

Worked example (advanced NSCLC second-line therapy DCE)

Decision context: oncologists and patient organizations want to know how patients trade efficacy against toxicity and administration burden for a second-line therapy.

1. Attribute development: qualitative interviews (n≈20) and literature reduce candidate attributes to five — median overall survival (levels: 6, 9, 12 months), grade 3–4 toxicity risk (10%, 25%, 40%), mode of administration (oral daily, IV every 3 weeks), serious immune-related adverse-event risk (1%, 5%, 10%), and monthly out-of-pocket cost ($50, $250, $600).
2. Design: a full factorial is 3×3×2×3×3 = 162 profiles; a D-efficient fractional design generates 24 paired-profile choice tasks, blocked into two versions of 12 tasks each, with a no-treatment opt-out and one dominated pair inserted as a rationality check.
3. Sample size: with c = 3, t = 12, a = 2, the Orme floor is 500·3/(12·2) ≈ 63 respondents per block — but to fit a mixed logit and latent classes we target n ≈ 300, recruited through thoracic-oncology clinics and a verified-diagnosis panel.

(4) Estimation: a conditional (multinomial) logit gives population mean part-worths; a random-parameters (mixed) logit with the cost coefficient held fixed and clinical attributes random captures preference heterogeneity and lets us derive the distribution of WTP. (5) Trade-off metrics: marginal WTP for a month of survival = (β_survival / |β_cost|) in dollars; maximum acceptable risk for a 3-month survival gain = (3·β_survival / β_toxicity). (6) Diagnostics: drop respondents who fail the dominated-pair check or straight-line, test IIA, compare conditional vs mixed logit by AIC/BIC and log-likelihood, inspect the sign/significance of part-worths against clinical expectation (monotonic in survival, decreasing in toxicity and cost), and report predicted choice shares only as conditional simulations, never as real-world uptake.

Decision diagram

flowchart TD
  Q[Decision question + target population] --> Attr[Qualitative attribute development<br/>interviews / literature -> 4-6 attributes, defined levels]
  Attr --> Des[D-efficient experimental design<br/>fractional factorial, blocking, opt-out, dominated-pair check]
  Des --> Field[Field the survey to the target frame<br/>screen diagnosis, randomize task order]
  Field --> Clean[Data quality screen<br/>drop straight-liners, speeders, rationality-check failures]
  Clean --> Est[Estimate choice model<br/>conditional logit -> mixed / latent-class logit]
  Est --> Out[Part-worths, attribute importance,<br/>WTP / maximum acceptable risk]
  Out --> Use[Benefit-risk, value frameworks, SDM tools<br/>report predicted shares ONLY as conditional simulations]

flowchart LR
  RUM["Random utility model<br/>U = V(attributes) + error"] --> Q1{Preference<br/>heterogeneity?}
  Q1 -- No --> CL["Conditional / multinomial logit<br/>population-mean part-worths, assumes IIA"]
  Q1 -- "Yes, continuous" --> ML["Mixed / random-parameters logit<br/>panel, distribution of part-worths"]
  Q1 -- "Yes, segments" --> LC[Latent-class logit<br/>discrete preference classes]
  ML --> Q2{Scale vs preference<br/>heterogeneity confounded?}
  LC --> Q2
  Q2 -- Yes --> GMNL[Generalized multinomial logit G-MNL<br/>separates scale from taste variance]
  CL --> WTP["Derive WTP = -beta_attr / beta_cost<br/>and maximum acceptable risk"]
  GMNL --> WTP

Worked example

Scenario

An HEOR analyst wants to know how patients with moderate-to-severe plaque psoriasis trade off efficacy, side-effect risk, dosing convenience, and monthly cost when choosing between two biologic treatment profiles. The analyst designs a simple DCE with four attributes and presents one example choice task to illustrate how the data are read.

Dataset

One DCE choice task — a respondent sees exactly this table and circles Profile A or Profile B.

Steps

Result

Patients value efficacy gains roughly 1.6 times more than they penalize an equivalent increase in infection risk; a jump from 55% to 75% clearance is worth accepting a 3-percentage-point higher annual infection risk, but only barely — and only if cost does not also rise. Monthly out-of-pocket cost is the strongest deterrent: patients would need a 20-percentage-point efficacy improvement to justify paying an extra $130/month.

Trade-offs

Runnable example

import numpy as np
import pandas as pd
from xlogit import MultinomialLogit, MixedLogit

# choices: long format, one row per alternative; `choice` is the 0/1 chosen flag within each (respondent_id, task_id).
choices = pd.read_parquet("dce_long.parquet")
choices["alt_key"] = choices["respondent_id"].astype(str) + "_" + choices["task_id"].astype(str)
varnames = ["survival", "toxicity", "oral", "cost"]   # design attributes (cost in $100s for stable scaling)

# (1) Conditional (multinomial) logit: population-mean part-worths under RUM/IIA.
cl = MultinomialLogit()
cl.fit(X=choices[varnames], y=choices["choice"], varnames=varnames,
       alts=choices["alt_id"], ids=choices["alt_key"])
cl.summary()

# Marginal willingness-to-pay = -beta_attr / beta_cost  (cost coded so a higher level lowers utility).
b = dict(zip(varnames, cl.coeff_))
wtp_per_survival_month = -b["survival"] / b["cost"] * 100.0   # undo the $100 scaling
print(f"WTP per extra month of survival: ${wtp_per_survival_month:,.0f}")

# (2) Mixed (random-parameters) logit: clinical attrs random, cost fixed; panels capture within-respondent correlation.
ml = MixedLogit()
ml.fit(X=choices[varnames], y=choices["choice"], varnames=varnames,
       alts=choices["alt_id"], ids=choices["alt_key"],
       panels=choices["respondent_id"],
       randvars={"survival": "n", "toxicity": "n", "oral": "n"},  # normal mixing; cost stays fixed
       n_draws=500)
ml.summary()   # mean and SD of each random part-worth quantify preference heterogeneity

library(idefix)
library(mlogit)

## (1) D-efficient design: 5 attributes, levels per the NSCLC example; 24 tasks blocked into 2 sets of 12, 2 alts.
lvls   <- c(3, 3, 2, 3, 3)                       # survival, toxicity, admin, ir-AE risk, cost
coding <- c("E", "E", "E", "E", "E")             # effects coding
set.seed(42)
des <- Modfed(cand.set = Profiles(lvls = lvls, coding = coding),
              n.sets = 24, n.alts = 2, alt.cte = c(0, 0),
              par.draws = rep(0, sum(lvls - 1)))  # zero priors -> D-optimal (use informative priors if available)
# des$design is the efficient choice design; block and field it, then attach respondents' choices.

## (2) Conditional logit on the fielded long data.
idx <- dfidx(dce, idx = list(c("task_id", "respondent_id"), "alt"), choice = "choice")
cl  <- mlogit(choice ~ survival + toxicity + oral + cost | 0, data = idx)
summary(cl)
wtp_survival <- -coef(cl)["survival"] / coef(cl)["cost"]   # marginal WTP per survival month
cat(sprintf("WTP per survival month: %.0f\n", wtp_survival))

## (3) Mixed logit (panel) for preference heterogeneity: clinical attrs random-normal, cost fixed.
ml <- mlogit(choice ~ survival + toxicity + oral + cost | 0, data = idx,
             rpar = c(survival = "n", toxicity = "n", oral = "n"),
             R = 500, panel = TRUE, correlation = FALSE)
summary(ml)   # estimated SDs flag attributes with heterogeneous preferences

/* (1) Build a D-efficient choice design: 5 attributes, then assess efficiency. */
%mktex(3 3 2 3 3, n=24, seed=42, out=cand);     /* candidate fractional-factorial design */
%choiceff(data=cand, model=class(x1 x2 x3 x4 x5 / sta),
          nsets=24, nalts=2, seed=42, out=design);   /* maximize D-efficiency for 24 sets x 2 alts */

/* (2) Conditional (multinomial) logit on the fielded long data: population-mean part-worths under RUM. */
proc mdc data=work.dce;
  id respondent_id;                              /* clusters repeated tasks within respondent */
  model choice = survival toxicity oral cost /
        type=clogit nchoice=2;                   /* 2 alternatives per choice set */
run;
/* Marginal WTP = -(beta_survival)/(beta_cost), computed from the parameter estimates above. */

/* (3) Mixed (random-parameters) logit: clinical attributes random-normal, cost fixed; Halton draws. */
proc mdc data=work.dce type=mixedlogit nmixed=500;
  id respondent_id;
  model choice = survival toxicity oral cost /
        type=mixedlogit nchoice=2;
  random survival toxicity oral / type=normal;   /* heterogeneous part-worths; cost left fixed */
run;

Citations

1. [1]Bridges JFP, Hauber AB, Marshall D, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value in Health. 2011;14(4):403-413.
2. [2]Hauber AB, González JM, Groothuis-Oudshoorn CGM, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR Conjoint Analysis Good Research Practices Task Force. Value in Health. 2016;19(4):300-315.

1. [3]Reed Johnson F, Lancsar E, Marshall D, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value in Health. 2013;16(1):3-13.

1. [4]Soekhai V, de Bekker-Grob EW, Ellis AR, Vass CM. Discrete choice experiments in health economics: past, present and future. PharmacoEconomics. 2019;37(2):201-226.