Credible Interval in Bayesian Statistics

In plain language

A credible interval is the Bayesian version of an uncertainty range. If a report says the 95% credible interval for a response rate is 19% to 42%, it means that, after combining the prior information with the observed data, 95% of the posterior probability for the response rate sits inside that range. It is not the same as a confidence interval, which describes long-run coverage of a repeated-sampling procedure.

What it is

A credible interval is the Bayesian uncertainty interval for a parameter. After the prior and likelihood have been combined into a posterior distribution, the analyst chooses an interval that contains a target amount of posterior probability, commonly 90%, 95%, or 99%. For a treatment response probability p, a 95% credible interval [0.19, 0.42] is read literally: conditional on the prespecified model, prior, and observed data, the posterior probability that p lies between 0.19 and 0.42 is 0.95. That conditional phrase is not boilerplate. If the prior is inappropriate, the likelihood is built from a biased outcome algorithm, or the model ignores site heterogeneity, the interval is a coherent summary of the wrong analysis.

The credible interval is distinct from a frequentist confidence interval. A 95% confidence interval is a property of a repeated-sampling procedure: before data are collected, 95% of intervals generated by that procedure would cover the fixed true value in repeated samples. Once a specific confidence interval has been computed, the fixed parameter either is or is not inside it. A 95% credible interval is instead a post-data probability statement about the parameter under the Bayesian model. This is the interpretation many clinical and HEOR readers casually give to confidence intervals, but it is only formally valid for a Bayesian interval.

Interval types and construction

The two common constructions are equal-tailed and highest posterior density (HPD, also called highest density interval). An equal-tailed 95% interval cuts 2.5% of posterior mass from each tail, so its bounds are the 0.025 and 0.975 posterior quantiles. It is easy to compute, easy to audit, and available from closed-form distributions such as the beta distribution. An HPD interval is the shortest interval containing 95% posterior mass; every point inside the interval has posterior density at least as high as every point outside it. HPD intervals are attractive for skewed posteriors but can be unstable or non-contiguous for multimodal posteriors. In production RWE work, use equal-tailed intervals for transparent tables unless the posterior is very skewed and the SAP prespecifies HPD reporting.

Credible intervals can come from closed-form posteriors or from posterior draws. In a beta-binomial model, a Beta(alpha, beta) posterior has exact quantiles from qbeta, scipy.stats.beta.ppf, or SAS QUANTILE("BETA", ...). In a Bayesian logistic, Cox, hierarchical, or network model, the interval is computed from MCMC draws after convergence has been checked. The credible interval is not valid if R-hat, effective sample size, divergent transitions, or trace plots indicate that the sampler did not explore the posterior.

Pros, cons, and trade-offs

- vs confidence interval: the credible interval answers the direct probability question that decision makers usually ask. The cost is that it depends on the prior and the likelihood model. In large samples with weak priors, credible and confidence intervals often look similar; in sparse RWE, rare disease, external control, or hierarchical analyses they can diverge. - Equal-tailed vs HPD: equal-tailed intervals are reproducible from quantiles and stable across software. HPD intervals can be shorter and more intuitive for skewed posteriors. Prefer equal-tailed for regulatory tables unless the analysis plan states otherwise. - Interval vs posterior probability threshold: a credible interval summarizes a central range, while a probability statement such as P(HR < 0.90 | data) answers a decision threshold directly. Use both when a payer, regulator, or data monitoring committee has a clinically meaningful threshold. - Weak prior vs informative prior: an informative prior can stabilize sparse data and borrow historical information, but it makes the interval depend on transportability of the prior evidence. Always include a prior sensitivity grid.

When to use

Use credible intervals whenever the analysis is Bayesian and the reader needs an uncertainty interval around an event rate, risk difference, log odds ratio, hazard ratio, regression coefficient, net monetary benefit, treatment ranking, or posterior predictive quantity. They are most useful in small samples, rare disease registries, pediatric extrapolation, Bayesian borrowing from historical controls, probabilistic sensitivity analysis, hierarchical site models, and network meta-analysis where the full posterior distribution is already being estimated.

When NOT to use - and when it is actively misleading

Do not label a frequentist confidence interval as a credible interval because it is easier to explain; that silently changes the inferential framework. Do not report a credible interval without naming the prior, likelihood, and interval construction. Do not use a credible interval to rescue a confounded design: the posterior probability is conditional on the model and does not convert an observational association into a causal effect. Do not report intervals from a non-converged MCMC model. Do not choose the prior after seeing the data to make the credible interval include or exclude a desired value.

Data-source operational depth

- Claims: the likelihood is usually built from algorithm-derived event counts or model coefficients. Endpoint misclassification, incomplete pharmacy or medical benefit capture, continuous-enrollment restrictions, and death/disenrollment censoring determine whether the interval summarizes the target endpoint or a claims-observed surrogate. If PPV is low, put misclassification uncertainty into the model or run a bias-analysis sensitivity rather than treating the count as adjudicated truth. - EHR: encounter-driven capture and out-of-network care make observed event counts incomplete. For Bayesian site-level models, include site random effects and posterior predictive checks for site outliers before interpreting pooled credible intervals. - Registry: registries often provide adjudicated endpoints but small denominators. Credible intervals are useful precisely because they remain interpretable in sparse data; the prior effective sample size must be reported so readers can see how much the prior pulled the interval. - Primary trial data: prespecify prior, threshold probabilities, and credible interval construction in the SAP. For regulatory use, also report operating characteristics under plausible data-generating scenarios. - Linked data: linked claims-EHR-registry data can support richer Bayesian likelihoods, but linkage selection and missingness should be modeled or sensitivity-tested; otherwise the interval is conditional on the linkable subset rather than the intended target population.

Worked example

A pediatric registry observes 14 responders among 40 treated patients. Historical evidence suggests a response probability near 20% with information roughly equivalent to 20 patients, encoded before analysis as Beta(4,16). The beta-binomial posterior is Beta(18,42). The posterior mean is 18/(18+42) = 0.30. The equal-tailed 95% credible interval is approximately [0.192, 0.421], and the posterior probability that the response probability exceeds a 25% decision threshold is approximately 0.798. The interpretation is not "this procedure covers the true response rate 95% of the time." It is: given the Beta(4,16) prior, the binomial likelihood from 14/40 responders, and the model assumptions, there is 95% posterior probability that the response probability lies between about 19.2% and 42.1%. A sensitivity analysis using a weak Jeffreys prior should accompany the primary interval so reviewers can see whether the conclusion is prior-driven.

Worked example

Scenario

A rare-disease pediatric registry has 14 responders among 40 treated patients. Before analyzing the registry, the team encoded historical evidence as a Beta(4,16) prior, centered at a 20% response rate with effective sample size 20. The question is the posterior response rate and whether the response probability plausibly exceeds a 25% decision threshold.

Dataset

Prior pseudo-counts and observed registry counts for the beta-binomial credible interval.

Steps

• Start with the prespecified prior Beta(4,16), which contributes 4 prior responders and 16 prior nonresponders.

• Add the observed registry counts: alpha_post = 4 + 14 = 18 and beta_post = 16 + 26 = 42.

• Compute the posterior mean as 18/(18+42) = 0.300, which lies between the historical prior mean (0.200) and the registry estimate (14/40 = 0.350).

• Read the equal-tailed 95% credible interval from the Beta(18,42) quantiles: approximately 0.192 to 0.421.

• Compute the threshold probability P(response probability > 0.25 | data) = 1 - CDF_Beta(0.25; 18,42), approximately 0.798.

Result

Posterior Beta(18,42), posterior mean 0.300, equal-tailed 95% credible interval approximately [0.192, 0.421], and P(response probability > 0.25 | data) approximately 0.798. The interval is a direct posterior probability statement conditional on the prior, data, and model.

Runnable example

import numpy as np
from scipy import stats

def beta_binomial_posterior(alpha0, beta0, responders, n):
    return alpha0 + responders, beta0 + (n - responders)

def equal_tailed_beta_cri(alpha_post, beta_post, level=0.95):
    tail = (1.0 - level) / 2.0
    dist = stats.beta(alpha_post, beta_post)
    return dist.ppf(tail), dist.ppf(1.0 - tail)

def hpd_from_draws(draws, level=0.95):
    draws = np.sort(np.asarray(draws))
    m = int(np.floor(level * draws.size))
    widths = draws[m:] - draws[:draws.size - m]
    j = int(np.argmin(widths))
    return draws[j], draws[j + m]

alpha_post, beta_post = beta_binomial_posterior(4, 16, responders=14, n=40)
lo, hi = equal_tailed_beta_cri(alpha_post, beta_post)
posterior = stats.beta(alpha_post, beta_post)

print(f"Posterior Beta({alpha_post}, {beta_post})")
print(f"Posterior mean: {alpha_post / (alpha_post + beta_post):.3f}")
print(f"95% equal-tailed credible interval: [{lo:.3f}, {hi:.3f}]")
print(f"P(response probability > 0.25 | data): {1 - posterior.cdf(0.25):.3f}")

# If you have MCMC draws for a non-conjugate model, compute quantile or HPD intervals from draws.
draws = posterior.rvs(size=20000, random_state=7)
print("95% HPD interval from draws:", tuple(round(x, 3) for x in hpd_from_draws(draws)))

beta_binomial_posterior <- function(alpha0, beta0, responders, n) {
  c(alpha = alpha0 + responders, beta = beta0 + (n - responders))
}

equal_tailed_beta_cri <- function(alpha_post, beta_post, level = 0.95) {
  tail <- (1 - level) / 2
  qbeta(c(tail, 1 - tail), alpha_post, beta_post)
}

pars <- beta_binomial_posterior(alpha0 = 4, beta0 = 16, responders = 14, n = 40)
cri <- equal_tailed_beta_cri(pars["alpha"], pars["beta"])
post_mean <- pars["alpha"] / sum(pars)
prob_above_25 <- 1 - pbeta(0.25, pars["alpha"], pars["beta"])

cat(sprintf("Posterior Beta(%g, %g)\n", pars["alpha"], pars["beta"]))
cat(sprintf("Posterior mean: %.3f\n", post_mean))
cat(sprintf("95%% equal-tailed credible interval: [%.3f, %.3f]\n", cri[1], cri[2]))
cat(sprintf("P(response probability > 0.25 | data): %.3f\n", prob_above_25))