Ethnographic / Observational Qualitative Study

In plain language

An ethnographic study answers "how and why does care actually happen here?" by having a researcher spend sustained time inside a real setting — a clinic, an infusion suite, a patient's home — watching what people do, talking with them about it on the spot, and taking detailed notes. It is about understanding meaning and context, not measuring how often something happens, so it never produces a rate, an effect size, or a percentage. Its signature strength is catching the gap between what people say they do and what they actually do, which a survey can never see. Its honest limit: a few richly studied sites tell you how a process works, not how common it is across a whole population.

An ethnographic / observational qualitative study generates primary qualitative data by placing the investigator inside the setting where care is delivered or experienced — a dialysis unit, an oncology infusion suite, a community pharmacy, a patient's home — over a sustained period. The defining tools are participant observation (the researcher watches, and often partly takes part in, routine activity), in-context (often semi-structured) interviewing, and analysis of field notes, documents, and artifacts. The output is not an effect estimate but a defensible, theory-informed account of how and why a phenomenon happens: why a newly launched biologic is barely used in community rheumatology, what "treatment burden" concretely means to a person on home dialysis, how a digital therapeutic is actually appropriated (or abandoned) on a real ward. In RWE/HEOR this is the method that surfaces the mechanisms, contexts, and patient-prioritized outcomes that a claims or EHR analysis can describe but never explain.

Core conceptual distinction

. The estimand-bearing methods in this catalog ask "what is the effect of A vs B?"; an ethnography asks "what is going on here, and why?" Its inferential target is analytic (theoretical) generalization to concepts and mechanisms, not statistical generalization to a population — a single richly studied site can validly refine a theory of treatment burden even though it estimates no parameter. Two further distinctions matter. (1) Ethnography vs the qualitative interview study (`qualitative-interview`): interviews capture accounts of behaviour; ethnography captures behaviour in situ and the gap between what people say and what they do — its signature contribution is observed practice, not reported practice. (2) Primary qualitative collection vs qualitative evidence synthesis (`qualitative-synthesis`): this concept is fieldwork that produces new data; synthesis aggregates existing qualitative studies. Rigour is judged not by p-values but by credibility, dependability, confirmability, and transferability (Lincoln & Guba's trustworthiness criteria), operationalized through reflexivity, an audit trail, and reporting against COREQ or SRQR — not by sample size or precision.

Pros, cons, and trade-offs

- vs the qualitative interview study (`qualitative-interview`): observation reveals tacit, routinized, or socially undesirable practice that interviews miss (e.g., how clinicians actually counsel on adherence vs how they describe it), and it does not depend on a participant's recall or candour. Cost: it is far more resource- and access-intensive, raises sharper reactivity (Hawthorne) and consent-of-the-observed problems, and yields data that are harder to anonymize. Prefer ethnography when the research question is about practice, process, and context; prefer interviews when it is about experience, meaning, and perspective that observation cannot reach. - vs quantitative RWE (claims/EHR cohorts): ethnography explains mechanism, generates hypotheses, and defines patient-relevant outcomes and burdens that structured data cannot represent. Cost: it cannot estimate incidence, effect size, or cost, and does not support statistical generalization. The two are complementary, not rival — see `mixed-methods` for principled integration (e.g., explaining a counter-intuitive utilization finding). - vs structured surveys / preference studies (`preference-study`): ethnography is open-ended and discovery-oriented, appropriate before the constructs are known. Cost: it cannot quantify prevalence or trade-offs. The standard sequence is ethnography/interviews to generate the conceptual model, then surveys/discrete-choice or COA instruments to measure it.

When to use

. The question is "how" or "why" rather than "how much"; the construct (e.g., treatment burden, an implementation barrier, a patient-prioritized outcome) is poorly understood and must be discovered, not measured; behaviour-in-context and the say–do gap are central; you are building the conceptual model that will anchor downstream PRO/COA development (`pro-development`) or a preference study; or you need implementation/context evidence to interpret a quantitative signal or to support an HTA submission's narrative on patient experience and unmet need. Use it as the hypothesis-generating front end of a mixed-methods program.

When NOT to use — and when it is actively misleading or dangerous

- When the question is comparative or quantitative. Ethnographic data cannot estimate effects, rates, or costs; presenting "most patients felt..." from a purposive sample of 12 as if it were a prevalence is a category error that invites decision-makers to read frequency into data that carry none. - When the deliverable demands statistical generalization. A single-site account does not transport to a population by counting; claiming it does is the qualitative analog of fabricating external validity. - When access forces a biased vantage point. Gatekeeper-controlled access (you see only the well-run clinic, the compliant patients) produces selection of the observed — the qualitative cousin of selection bias — and the resulting account is confidently wrong about the typical case. - When reactivity cannot be managed. If the mere presence of an observer would so distort the behaviour of interest (e.g., a regulator-sensitive dispensing practice) that nothing authentic remains, observation is the wrong tool. - When it is deployed as decorative "patient voice." A few quotes bolted onto a quantitative dossier without a coding frame, reflexivity, saturation logic, or an audit trail is not evidence; it lends false authority and is rejected by serious HTA reviewers (e.g., NICE) who expect a transparent, appraisable qualitative method.

Data-source operational depth

. The "data sources" of an ethnography are its modes of primary collection, each with characteristic failure modes and workarounds. - Participant observation (the core mode): richest for practice and context. Failure modes: reactivity / Hawthorne (people perform when watched) — mitigate with prolonged engagement so behaviour normalizes and by triangulating observation against records and interviews; observer-role drift (the "observer-as-participant" slides toward "going native," losing analytic distance) — mitigate with reflexive memos and peer debriefing; consent of the observed in busy clinical space where bystanders cannot all consent — mitigate with site-level governance, posted notices, and field-note redaction. - In-context / semi-structured interviews: capture meaning and the participant's framing on the spot. Failure modes: recall and social-desirability bias in retrospective accounts, and interviewer effects — mitigate by anchoring questions to just-observed events and by reflexive bracketing of the interviewer's assumptions. - Focus groups: efficient for surfacing shared norms and disagreement. Failure modes: dominant-voice and conformity effects suppress minority experience and over-state consensus — mitigate with skilled facilitation and by not treating group consensus as individual prevalence. - Documentary / artifact analysis (protocols, patient diaries, device logs, posters): grounds claims in material traces. Failure mode: documents record the intended process, not the enacted one — read against observed practice. - Rapid / focused ethnography (compressed fieldwork for time-bound HEOR/implementation questions): pragmatic and fundable. Failure mode: premature claims of saturation when conceptual categories are still incomplete — pre-specify a saturation logic, document where new data stop adding new themes, and flag residual thinness honestly. - Linkage to structured RWE: in mixed-methods designs, qualitative findings are explicitly tied to a claims/EHR signal (see `mixed-methods`, `linked-data`). Workaround for the integration: keep an audit trail mapping each qualitative theme to the quantitative finding it explains, so the joint inference is appraisable rather than rhetorical.

Worked applied example

A manufacturer's commercial + Medicare FFS claims analysis shows that a newly launched subcutaneous biologic for moderate-to-severe rheumatoid arthritis has surprisingly low and slow uptake in community rheumatology versus academic centres, and that early initiators discontinue within 90 days more often than the trial would predict — a pattern the structured data can describe but not explain, and which threatens both an HTA value story and the brand's launch plan. A focused ethnography is commissioned to explain the mechanism. (1) Question: why is initiation low and early discontinuation high in community practice, and what would change it? (2) Sampling: purposive, theory-driven across four community practices selected for maximum variation (urban/rural, high/low biologic volume), plus negative cases (practices with high uptake) to test emerging explanations — not a random or convenience sample. (3) Collection: ~6 weeks per site of participant observation of prescribing visits, nurse-led injection training, and prior-authorization workflow; semi-structured interviews with rheumatologists, infusion/injection nurses, and patients who initiated, declined, or discontinued; documentary analysis of the prior-auth packets and patient starter-kit materials. (4) Time and saturation: collection continues until new observations and interviews stop generating new conceptual categories (theoretical saturation), documented explicitly rather than fixed in advance. (5) Analysis: field notes and transcripts are coded against a developing framework (a framework-analysis matrix: participants × analytic themes), with two independent coders on a defined coding sheet and a reported intercoder agreement on a sample of segments; reflexive memos record how the analyst's prior assumptions are challenged by negative cases. (6) Findings and use: the account identifies the binding constraint — prior-authorization friction and inadequate in-clinic injection-training capacity in community settings, not patient preference — yielding (a) a testable hypothesis for a follow-up quantitative analysis of discontinuation by prior-auth turnaround time, (b) a patient-prioritized treatment-burden construct that seeds a PRO/COA instrument (`pro-development`), and (c) implementation/context evidence for the HTA dossier. (7) Rigour and reporting: the study is written up against COREQ/SRQR with an audit trail, reflexivity statement, and explicit transferability claims (to comparable community settings, not to all RA patients), so an HTA reviewer can appraise it as evidence rather than anecdote.

Worked example

Scenario

A researcher spends several weeks inside a community rheumatology clinic to understand why a newly launched injectable biologic is barely being started, even though the medicine is available. Instead of handing out a survey, they sit in on prescribing visits, nurse injection-training sessions, and the back-office insurance-approval work, writing down what actually happens. The goal is not to count anything but to explain the process: what gets in the way of starting this drug, in this place, in real life.

Dataset

A handful of raw field-note entries an ethnographer would actually jot down — each is one observed moment, not a number. The analyst later groups these recurring moments into themes.

Steps

• Each row is a single thing the researcher actually witnessed, written down in plain detail — not a survey answer and not a tally.

• The researcher reads back over many such notes and notices the same situations recurring; she gives each recurring pattern a short name, which becomes a theme (the right-hand column).

• Repetition is what makes a theme trustworthy: "paperwork delays" appears in the exam room, the back office, and the waiting room, from different people on different days, so it is a real feature of the place rather than one person's bad day.

• Watching in person catches what a survey would miss — the last row shows a patient who says she is confident injecting but is observed hesitating, which is the say-do gap a questionnaire could never reveal.

• The researcher keeps observing until new visits stop producing new themes (saturation), which is how she knows the picture is reasonably complete — not because she hit a target sample size.

Result

The synthesized insight: starting this drug is being blocked mainly by slow insurance paperwork and a shortage of staff trained to teach injections — not by patients refusing the medicine. That explanation, grounded in repeated first-hand observation across several parts of the clinic, is something the prescribing and claims data could flag as "low uptake" but never explain. It points to fixable process problems and gives the team a real-world account of why patients drop off, with no statistics invented or implied.

Runnable example

import pandas as pd

def framework_matrix(coded: pd.DataFrame) -> pd.DataFrame:
    """Participant x theme matrix of segment counts; the spine of framework analysis."""
    matrix = (coded
              .pivot_table(index="participant_id", columns="theme",
                           values="segment_text", aggfunc="count", fill_value=0))
    return matrix

def theme_grounding(coded: pd.DataFrame) -> pd.DataFrame:
    """For each theme: how many distinct participants and which collection modes support it.
    Thin grounding (few participants, single mode) flags themes that are not yet saturated
    or rest on a single vantage point (e.g., observation only, never corroborated in interview)."""
    g = (coded.groupby("theme")
              .agg(n_segments=("segment_text", "count"),
                   n_participants=("participant_id", "nunique"),
                   modes=("source_mode", lambda s: ", ".join(sorted(s.unique()))))
              .sort_values("n_participants", ascending=False))
    return g

library(data.table)
library(irr)

coding_reliability <- function(coded) {
  setDT(coded)
  # Cohen's kappa on the two coders' assigned codes for the double-coded segments.
  ratings <- coded[, .(code_coder1, code_coder2)]
  k <- kappa2(ratings, weight = "unweighted")

  # Theme grounding: distinct participants and collection modes supporting each agreed code.
  agreed <- coded[code_coder1 == code_coder2]
  grounding <- agreed[, .(n_segments = .N,
                          n_participants = uniqueN(participant_id),
                          modes = paste(sort(unique(source_mode)), collapse = ", ")),
                      by = .(code = code_coder1)][order(-n_participants)]

  list(kappa = k$value, p_value = k$p.value, grounding = grounding)
}