Mixed-Methods Study

In plain language

A mixed-methods study collects two kinds of evidence on purpose: numbers that tell you how much or how often (the quantitative strand), and people's words and experiences that tell you why or how (the qualitative strand). The key word is integration — the two strands are designed from the start to inform each other, not just exist side by side in the same report. A common pattern in drug research is to use medical records or insurance claims to find which patients did poorly, then follow up with interviews to understand the reasons — producing an insight that neither source could reach alone.

A mixed-methods study deliberately combines a quantitative (QUANT) strand and a qualitative (QUAL) strand and — critically — integrates them so the whole exceeds the sum of the parts. In real-world evidence and HEOR the QUANT strand is typically a claims/EHR/registry analysis (incidence, comparative effectiveness, adherence, cost, HCRU) and the QUAL strand is interviews, focus groups, ethnographic observation, or structured analysis of clinical free text. The design's defining act is integration — at the level of design, methods, and interpretation — not the simple presence of two data types side by side.

Core conceptual distinction

. What separates mixed methods from "we also did some interviews" is purposeful integration and an explicit priority and sequence. Three canonical designs (Creswell/Plano Clark typology) map directly onto RWE workflows: (1) Convergent (parallel) — QUANT and QUAL collected concurrently and merged to corroborate or explain divergence (e.g., a claims adherence analysis run alongside patient interviews, then placed in a joint display). (2) Explanatory sequential (QUANT -> QUAL) — a quantitative result is explained by a follow-on qualitative strand; the QUANT result drives the qualitative sampling frame (e.g., claims identify low-adherence patients, who are then purposively sampled for interviews). (3) Exploratory sequential (QUAL -> QUANT) — qualitative work builds an instrument or hypothesis that is then measured at scale (e.g., interviews surface burden domains that become a PRO later validated in registry data). The integration point — connecting (sampling one strand from another), building (one strand creates the instrument for the other), or merging (joint display + meta-inference) — is the unit of methodological rigor (Fetters 2013). A joint display that simply parks a quantitative confidence interval next to a qualitative theme with no cross-talk is not integration; it is two studies in one document. The HEOR-relevant special case is the effectiveness-implementation hybrid (Curran 2012), which blends a comparative-effectiveness QUANT estimand with QUAL/process data on adoption, reach, and fidelity, accelerating the move from evidence to uptake.

Pros, cons, and trade-offs

(specific and comparative). - vs a stand-alone quantitative RWE study (e.g., claims comparative effectiveness): mixed methods adds the why and for whom — mechanism, context, patient/clinician meaning, and reasons for heterogeneity that a hazard ratio cannot supply. Cost: roughly doubles the design surface, requires dual expertise, and lengthens timelines; if the QUANT question is precise and mechanism is already understood, the QUAL strand is overhead. Prefer mixed methods when an effect estimate alone will not change behavior or policy because decision-makers need to understand implementation, acceptability, or unexplained heterogeneity. - vs a stand-alone qualitative study: the QUANT strand anchors prevalence, magnitude, and generalizable structure so qualitative themes are not over-extrapolated. Cost: the qualitative depth can be diluted when forced to serve a fixed quantitative frame. Prefer mixed methods when you need both representativeness and depth. - vs simply running two separate studies: integration is the whole value proposition; a true mixed-methods design plans the connect/build/merge point a priori and reports meta-inferences. Cost: integration is the hardest, most failure-prone step and demands governance (a single protocol, aligned timelines, an integration analyst). Do not label two parallel but unconnected studies "mixed methods." - vs a pragmatic trial or hybrid trial that bolts on a process evaluation: in RWE the QUANT strand is usually observational, which is cheaper and faster than a trial but inherits all observational confounding; the QUAL strand cannot fix biased point estimates. Prefer a hybrid trial when you can randomize; prefer observational mixed methods when randomization is infeasible and you still need mechanism + magnitude.

When to use

. Use mixed methods when (a) a quantitative finding is surprising, heterogeneous, or null and you need qualitative explanation (explanatory sequential); (b) you must develop or content-validate a PRO/instrument before fielding it (exploratory sequential); (c) you need corroboration across data types for a high-stakes claim (convergent); (d) you are studying implementation, adoption, or de-adoption of a therapy/program where uptake and fidelity matter as much as effect (effectiveness-implementation hybrid); or (e) a payer/HTA/regulatory audience has explicitly asked "we believe the number, now tell us why and whether it will hold in practice."

When NOT to use — and when it is actively misleading or dangerous

. - When the decision turns on a single precise estimate and mechanism is settled. A bolted-on QUAL strand wastes resources and can dilute a clean comparative-effectiveness message; reviewers may read the qualitative material as the study hedging an unconvincing primary result. - When integration is not actually planned. The most common and most dangerous failure is the "QUAL fig leaf": a handful of unsystematic quotes used to illustrate the quantitative result post hoc, presented as if it were triangulated evidence. This launders selection-biased anecdote into the authority of a number and is worse than reporting the QUANT alone. - When the qualitative sample is silently selected by the quantitative strand and that selection is ignored. If you purposively sample interviewees from a claims-defined subgroup (e.g., persistent users), their narratives do not generalize to discontinuers — treating them as the patient voice is a selection error masquerading as depth. - When divergence between strands is buried instead of interrogated. If the interviews contradict the claims result, that discordance is the finding; suppressing it to present a tidy convergent story is misleading. - When timelines or budgets force a degenerate strand. Three rushed interviews or an under-powered survey appended to a rigorous claims analysis is not mixed methods — it is a weak study with a misleading label.

Data-source operational depth

. - Claims (FFS vs MA vs commercial): Excellent as the QUANT sampling frame and denominator (continuous enrollment, `fill_date`, `days_supply`, PDC quintiles, HCRU, cost), but claims carry no patient voice — the QUAL strand must be recruited externally, which requires consent/contact infrastructure that selects on engagement, literacy, and willingness. Linking a claims cohort to interviewable patients typically routes through a health plan or provider, introducing site/plan selection. Medicare Advantage-only person-time lacks FFS claims, so a sampling frame built on claims completeness silently excludes MA enrollees; document which population your QUAL inferences actually cover. Failure mode: sampling interviewees only from members reachable by the plan and generalizing to "patients." - EHR: Free-text notes and NLP can substitute for a primary qualitative strand at scale (e.g., theme extraction from clinician notes), trading depth and the patient's own framing for volume; notes capture the clinician's construction of the encounter, not the patient's lived experience. EHR also supports recruiting real interviewees (contact info, active patients), but visit-driven capture means patients who leave the system are differentially absent from both strands. Failure mode: passing NLP-coded note themes off as qualitative integration without acknowledging the clinician-mediated lens. - Registry: Often pre-bundles PROs and patient-reported context, which is convenient but conflates measurement (a fixed instrument captured prospectively) with integration (linking that signal to an independent qualitative inquiry). Registries are strong for adjudicated outcomes and disease severity that can stratify purposive sampling. Failure mode: reporting registry PRO scores beside clinical outcomes and calling it mixed methods when no qualitative strand or meta-inference exists. - Linked claims-EHR-registry-PRO: The ideal substrate — claims completeness + EHR depth + registry adjudication + a recruitable, consented population for QUAL — but linkage selects on the linkable subset and creates date-reconciliation problems across order/fill/service/survey dates that must be resolved before a strand can be sampled from another. Failure mode: assuming the linked, consented subset represents the source population for qualitative inferences.

Worked example (explanatory sequential, claims QUANT -> QUAL)

Question: why do roughly a third of initiators of a chronic oral therapy become poorly adherent, and what does that look like in claims? (1) QUANT strand (claims). Build a new-user cohort with 365 days of continuous medical+pharmacy enrollment before the first qualifying `fill_date`; require no fill of the drug class in the washout. Over a 12-month landmark, compute proportion of days covered (PDC) from `fill_date` + `days_supply`, capping overlap per stockpiling rules. Classify PDC into quintiles; flag the bottom quintile (PDC < 0.40) as poorly adherent and the top quintile (PDC >= 0.90) as highly adherent, excluding MA-only person-time so adherence is measured on observable fills. (2) Connect (the integration point). Use the QUANT result to drive stratified purposeful sampling: draw ~N=12 from the bottom quintile and ~N=12 from the top quintile (sampling to thematic saturation, not statistical power), recruited through the contributing plan with documented consent — and record that this routes only through reachable, consenting members. (3) QUAL strand. Semi-structured interviews on access, side effects, cost, beliefs, and the texture of daily dosing; code thematically. (4) Merge / meta-inference (joint display). Lay claims-observable patterns (early `days_supply` gaps, switching, 90-day mail-order vs 30-day retail, cost-sharing tier) against interview themes in a single joint display, explicitly noting concordance (gappy fill patterns <-> reported cost barriers) and discordance (apparently adherent fill records <-> reported pill-skipping = "primary refill, secondary non-ingestion," invisible to claims). The meta-inference — that claims under-detect a behavioral non-adherence phenotype concentrated among cost-strained patients — is the deliverable that neither strand produced alone and that directly informs an adherence intervention. Report the QUAL sampling selection (plan-reachable, consenting) as a bound on transportability.

Worked example

Scenario

A research team wants to understand why one in three patients who start a daily oral medication for a chronic condition stop filling it within a year. They use pharmacy claims to measure how consistently each patient refilled, then recruit patients from the most- and least-consistent groups for phone interviews. The explanatory sequential design means the numbers come first and steer who gets interviewed.

Dataset

Mapping the research question across both strands and their integration point

Steps

• Run the quantitative strand first: pull all patients who started the medication with no prior fills in the look-back period, then calculate their refill consistency score over the next 12 months using each fill date and how many days that fill was supposed to last.

• Split patients into two groups based on their scores: a low group (filled less than 40 percent of the year) and a high group (filled 90 percent or more of the year).

• Use those two groups as the sampling frame for the qualitative strand: recruit roughly 12 patients from each group through the health plan, which requires consent and means only reachable, willing patients are included.

• Conduct semi-structured phone interviews asking about access, cost, side effects, daily routines, and their own sense of how well they took the medication.

• Code the interview transcripts into themes, then build the joint display: place each interviewed patient's refill score next to their top interview theme in a single table.

• Read across the joint display for concordance (gappy scores match cost-barrier theme — the two strands agree) and discordance (high scores but the patient reports skipping doses — the claims missed this entirely).

• State the meta-inference: the integrated finding is that claims-visible refilling and actual ingestion can come apart, and the patients who look adherent in the data but are silently skipping doses represent a distinct group that would be missed by any intervention designed from numbers alone.

Result

The integrated insight is that refill records and actual pill-taking diverge for a subgroup of patients who pick up their prescriptions on time but skip doses because of tolerable but persistent side effects. Neither the claims data (which only sees fills, not ingestion) nor the interviews alone (which cannot show how representative this group is) could surface this finding. The joint display makes the blind spot visible and points directly at a new intervention target.

Runnable example

import pandas as pd
import numpy as np

LOW_PDC, HIGH_PDC = 0.40, 0.90      # bottom / top adherence strata from the QUANT landmark
N_PER_STRATUM = 12                  # sample to saturation, not statistical power

def select_purposive_sample(cohort: pd.DataFrame, seed: int = 1) -> pd.DataFrame:
    # Eligible for QUAL = observable adherence (drop MA-only) AND a real consent/contact path.
    elig = cohort[(~cohort["ma_only"]) & (cohort["reachable"])].copy()
    elig["stratum"] = np.where(elig["pdc"] < LOW_PDC, "poorly_adherent",
                        np.where(elig["pdc"] >= HIGH_PDC, "highly_adherent", "middle"))
    extremes = elig[elig["stratum"].isin(["poorly_adherent", "highly_adherent"])]
    # Stratified purposive draw; selection is plan-reachable + consenting -> a transportability bound, not "patients."
    sample = (extremes.groupby("stratum", group_keys=False)
                      .apply(lambda g: g.sample(min(len(g), N_PER_STRATUM), random_state=seed)))
    return sample[["person_id", "stratum", "pdc", "cost_share_tier", "switched", "early_gap_days"]]

def build_joint_display(sample: pd.DataFrame, themes: pd.DataFrame) -> pd.DataFrame:
    # Merge claims-observable patterns (QUANT) against coded interview themes (QUAL) per person.
    top_theme = (themes.sort_values("salience", ascending=False)
                       .groupby("person_id").first().reset_index())
    jd = sample.merge(top_theme, on="person_id", how="left")
    # Concordance: does the dominant qualitative theme line up with the claims-observable signal?
    jd["concordant"] = np.where(
        (jd["stratum"] == "poorly_adherent") & (jd["theme"] == "cost_barrier") & (jd["cost_share_tier"] >= 3),
        True,
        np.where((jd["stratum"] == "highly_adherent") & (jd["theme"] == "routine_established"), True, False),
    )
    # Discordance to interrogate: claims look adherent but interviews report pill-skipping (invisible to claims).
    jd["claims_blind_spot"] = (jd["stratum"] == "highly_adherent") & (jd["theme"] == "skips_doses")
    return jd

library(data.table)
LOW_PDC <- 0.40; HIGH_PDC <- 0.90
N_PER_STRATUM <- 12L

select_purposive_sample <- function(cohort, seed = 1L) {
  setDT(cohort)
  elig <- cohort[!ma_only & reachable]                       # observable adherence + a consent/contact path
  elig[, stratum := fifelse(pdc < LOW_PDC, "poorly_adherent",
                      fifelse(pdc >= HIGH_PDC, "highly_adherent", "middle"))]
  ext <- elig[stratum %chin% c("poorly_adherent", "highly_adherent")]
  set.seed(seed)
  # Stratified purposive draw; selection = plan-reachable + consenting (a transportability bound).
  sample <- ext[, .SD[sample(.N, min(.N, N_PER_STRATUM))], by = stratum]
  sample[, .(person_id, stratum, pdc, cost_share_tier, switched, early_gap_days)]
}

build_joint_display <- function(sample, themes) {
  setDT(sample); setDT(themes)
  setorder(themes, -salience)
  top_theme <- themes[, .SD[1L], by = person_id]
  jd <- merge(sample, top_theme, by = "person_id", all.x = TRUE)
  # Concordance between the QUANT stratum and the dominant QUAL theme.
  jd[, concordant := (stratum == "poorly_adherent" & theme == "cost_barrier" & cost_share_tier >= 3) |
                     (stratum == "highly_adherent" & theme == "routine_established")]
  # Claims blind spot: looks adherent in claims, but interviews report dose-skipping.
  jd[, claims_blind_spot := stratum == "highly_adherent" & theme == "skips_doses"]
  jd[]
}