CONCEPTINTERMEDIATEPYTHON · R · SAS5 citations

Credible Interval in Bayesian Statistics

A Bayesian interval estimate containing a specified share of the posterior distribution for an unknown parameter, so a 95% credible interval means P(parameter is inside the interval | data, model, and prior) = 0.95.

Inferential Statisticsbayesiancredible-intervalposterior-intervalequal-tailed-intervalhpd-intervalposterior-probabilityuncertainty-quantificationrare-disease

On this page

Methods reference only. Use primary source citations and local policy before applying this in a study protocol, regulatory submission, payer dossier, or clinical decision.

In plain language

A credible interval is the Bayesian version of an uncertainty range. If a report says the 95% credible interval for a response rate is 19% to 42%, it means that, after combining the prior information with the observed data, 95% of the posterior probability for the response rate sits inside that range. It is not the same as a confidence interval, which describes long-run coverage of a repeated-sampling procedure.

When to use it

—Prefer credible intervals for Bayesian analyses, sparse data, historical borrowing, hierarchical models, and decision problems needing posterior probabilities.

—Report both: the credible interval for uncertainty and threshold probabilities for decision rules.

—Use credible intervals for current parameters; add prediction intervals when between-site or between-study heterogeneity matters to transportability.

Watch out for

—Requires explicit prior and likelihood choices, plus prior sensitivity analysis. It does not guarantee frequentist coverage unless the Bayesian procedure is calibrated for the design.

—A single interval can hide whether most posterior mass is near a clinically trivial effect or near a decision threshold.

—Does not by itself describe where a future site's or future study's effect may lie; hierarchical and NMA settings often need posterior predictive intervals as well.

What it is

A credible interval is the Bayesian uncertainty interval for a parameter. After the prior and likelihood have been combined into a posterior distribution, the analyst chooses an interval that contains a target amount of posterior probability, commonly 90%, 95%, or 99%. For a treatment response probability p, a 95% credible interval [0.19, 0.42] is read literally: conditional on the prespecified model, prior, and observed data, the posterior probability that p lies between 0.19 and 0.42 is 0.95. That conditional phrase is not boilerplate. If the prior is inappropriate, the likelihood is built from a biased outcome algorithm, or the model ignores site heterogeneity, the interval is a coherent summary of the wrong analysis.

The credible interval is distinct from a frequentist confidence interval. A 95% confidence interval is a property of a repeated-sampling procedure: before data are collected, 95% of intervals generated by that procedure would cover the fixed true value in repeated samples. Once a specific confidence interval has been computed, the fixed parameter either is or is not inside it. A 95% credible interval is instead a post-data probability statement about the parameter under the Bayesian model. This is the interpretation many clinical and HEOR readers casually give to confidence intervals, but it is only formally valid for a Bayesian interval.

Interval types and construction

The two common constructions are equal-tailed and highest posterior density (HPD, also called highest density interval). An equal-tailed 95% interval cuts 2.5% of posterior mass from each tail, so its bounds are the 0.025 and 0.975 posterior quantiles. It is easy to compute, easy to audit, and available from closed-form distributions such as the beta distribution. An HPD interval is the shortest interval containing 95% posterior mass; every point inside the interval has posterior density at least as high as every point outside it. HPD intervals are attractive for skewed posteriors but can be unstable or non-contiguous for multimodal posteriors. In production RWE work, use equal-tailed intervals for transparent tables unless the posterior is very skewed and the SAP prespecifies HPD reporting.

Credible intervals can come from closed-form posteriors or from posterior draws. In a beta-binomial model, a Beta(alpha, beta) posterior has exact quantiles from qbeta, scipy.stats.beta.ppf, or SAS QUANTILE("BETA", ...). In a Bayesian logistic, Cox, hierarchical, or network model, the interval is computed from MCMC draws after convergence has been checked. The credible interval is not valid if R-hat, effective sample size, divergent transitions, or trace plots indicate that the sampler did not explore the posterior.

Pros, cons, and trade-offs

vs confidence interval: the credible interval answers the direct probability question that decision makers usually ask. The cost is that it depends on the prior and the likelihood model. In large samples with weak priors, credible and confidence intervals often look similar; in sparse RWE, rare disease, external control, or hierarchical analyses they can diverge.
Equal-tailed vs HPD: equal-tailed intervals are reproducible from quantiles and stable across software. HPD intervals can be shorter and more intuitive for skewed posteriors. Prefer equal-tailed for regulatory tables unless the analysis plan states otherwise.
Interval vs posterior probability threshold: a credible interval summarizes a central range, while a probability statement such as P(HR < 0.90 | data) answers a decision threshold directly. Use both when a payer, regulator, or data monitoring committee has a clinically meaningful threshold.
Weak prior vs informative prior: an informative prior can stabilize sparse data and borrow historical information, but it makes the interval depend on transportability of the prior evidence. Always include a prior sensitivity grid.

When NOT to use - and when it is actively misleading

Do not label a frequentist confidence interval as a credible interval because it is easier to explain; that silently changes the inferential framework. Do not report a credible interval without naming the prior, likelihood, and interval construction. Do not use a credible interval to rescue a confounded design: the posterior probability is conditional on the model and does not convert an observational association into a causal effect. Do not report intervals from a non-converged MCMC model. Do not choose the prior after seeing the data to make the credible interval include or exclude a desired value.

Data-source operational depth

Claims: the likelihood is usually built from algorithm-derived event counts or model coefficients. Endpoint misclassification, incomplete pharmacy or medical benefit capture, continuous-enrollment restrictions, and death/disenrollment censoring determine whether the interval summarizes the target endpoint or a claims-observed surrogate. If PPV is low, put misclassification uncertainty into the model or run a bias-analysis sensitivity rather than treating the count as adjudicated truth.
EHR: encounter-driven capture and out-of-network care make observed event counts incomplete. For Bayesian site-level models, include site random effects and posterior predictive checks for site outliers before interpreting pooled credible intervals.
Registry: registries often provide adjudicated endpoints but small denominators. Credible intervals are useful precisely because they remain interpretable in sparse data; the prior effective sample size must be reported so readers can see how much the prior pulled the interval.
Primary trial data: prespecify prior, threshold probabilities, and credible interval construction in the SAP. For regulatory use, also report operating characteristics under plausible data-generating scenarios.
Linked data: linked claims-EHR-registry data can support richer Bayesian likelihoods, but linkage selection and missingness should be modeled or sensitivity-tested; otherwise the interval is conditional on the linkable subset rather than the intended target population.

Worked example

A pediatric registry observes 14 responders among 40 treated patients. Historical evidence suggests a response probability near 20% with information roughly equivalent to 20 patients, encoded before analysis as Beta(4,16). The beta-binomial posterior is Beta(18,42). The posterior mean is 18/(18+42) = 0.30. The equal-tailed 95% credible interval is approximately [0.192, 0.421], and the posterior probability that the response probability exceeds a 25% decision threshold is approximately 0.798. The interpretation is not "this procedure covers the true response rate 95% of the time." It is: given the Beta(4,16) prior, the binomial likelihood from 14/40 responders, and the model assumptions, there is 95% posterior probability that the response probability lies between about 19.2% and 42.1%. A sensitivity analysis using a weak Jeffreys prior should accompany the primary interval so reviewers can see whether the conclusion is prior-driven.

Decision diagram

flowchart TD
  Prior[Prespecified prior<br/>Beta(4,16)] --> Posterior[Posterior<br/>Beta(18,42)]
  Data[Registry data<br/>14 responders / 40 patients] --> Posterior
  Posterior --> ET[Equal-tailed 95% credible interval<br/>posterior quantiles 0.025 and 0.975]
  Posterior --> HPD[HPD interval<br/>shortest 95% posterior mass]
  Posterior --> Threshold[Decision probability<br/>P(response > 0.25 | data)]
  ET --> Report[Report interval type, prior, likelihood,<br/>sensitivity analysis, convergence if MCMC]
  Threshold --> Report

Credible interval workflow for a beta-binomial response-rate analysis. The interval is read from the posterior distribution and should be reported with the prior, likelihood, and threshold probability.

Worked example

Scenario

A rare-disease pediatric registry has 14 responders among 40 treated patients. Before analyzing the registry, the team encoded historical evidence as a Beta(4,16) prior, centered at a 20% response rate with effective sample size 20. The question is the posterior response rate and whether the response probability plausibly exceeds a 25% decision threshold.

Dataset

Prior pseudo-counts and observed registry counts for the beta-binomial credible interval.

source	responders	nonresponders	total_information	response_rate
prior_Beta_4_16	4	16	20	0.2
registry_data	14	26	40	0.35
posterior_Beta_18_42	18	42	60	0.3

Steps

1Start with the prespecified prior Beta(4,16), which contributes 4 prior responders and 16 prior nonresponders.

2Add the observed registry counts: alpha_post = 4 + 14 = 18 and beta_post = 16 + 26 = 42.

3Compute the posterior mean as 18/(18+42) = 0.300, which lies between the historical prior mean (0.200) and the registry estimate (14/40 = 0.350).

4Read the equal-tailed 95% credible interval from the Beta(18,42) quantiles: approximately 0.192 to 0.421.

5Compute the threshold probability P(response probability > 0.25 | data) = 1 - CDF_Beta(0.25; 18,42), approximately 0.798.

Result

Posterior Beta(18,42), posterior mean 0.300, equal-tailed 95% credible interval approximately [0.192, 0.421], and P(response probability > 0.25 | data) approximately 0.798. The interval is a direct posterior probability statement conditional on the prior, data, and model.

Trade-offs

vs. Frequentist confidence interval

Pros of this

—Provides a direct probability statement about the parameter conditional on the model, prior, and data. This is often the interpretation readers want when they see an interval estimate.

vs. Posterior probability of benefit only

Pros of this

—Shows the range of plausible parameter values, not only the probability of crossing one threshold.

vs. Prediction interval

Pros of this

—Summarizes uncertainty about the parameter estimated in the current model.

Runnable example

Exact equal-tailed credible intervals for a beta-binomial response rate, plus a simple HPD interval from posterior draws. The example reproduces the worked example: prior Beta(4,16), 14 responses in 40 registry patients, posterior Beta(18,42).

requires: numpy · scipy

import numpy as np
from scipy import stats

def beta_binomial_posterior(alpha0, beta0, responders, n):
    return alpha0 + responders, beta0 + (n - responders)

def equal_tailed_beta_cri(alpha_post, beta_post, level=0.95):
    tail = (1.0 - level) / 2.0
    dist = stats.beta(alpha_post, beta_post)
    return dist.ppf(tail), dist.ppf(1.0 - tail)

def hpd_from_draws(draws, level=0.95):
    draws = np.sort(np.asarray(draws))
    m = int(np.floor(level * draws.size))
    widths = draws[m:] - draws[:draws.size - m]
    j = int(np.argmin(widths))
    return draws[j], draws[j + m]

alpha_post, beta_post = beta_binomial_posterior(4, 16, responders=14, n=40)
lo, hi = equal_tailed_beta_cri(alpha_post, beta_post)
posterior = stats.beta(alpha_post, beta_post)

print(f"Posterior Beta({alpha_post}, {beta_post})")
print(f"Posterior mean: {alpha_post / (alpha_post + beta_post):.3f}")
print(f"95% equal-tailed credible interval: [{lo:.3f}, {hi:.3f}]")
print(f"P(response probability > 0.25 | data): {1 - posterior.cdf(0.25):.3f}")

# If you have MCMC draws for a non-conjugate model, compute quantile or HPD intervals from draws.
draws = posterior.rvs(size=20000, random_state=7)
print("95% HPD interval from draws:", tuple(round(x, 3) for x in hpd_from_draws(draws)))

Base R implementation for beta-binomial credible intervals and posterior threshold probabilities. Uses qbeta and pbeta, so the result is exact for the conjugate model and does not require MCMC.

beta_binomial_posterior <- function(alpha0, beta0, responders, n) {
  c(alpha = alpha0 + responders, beta = beta0 + (n - responders))
}

equal_tailed_beta_cri <- function(alpha_post, beta_post, level = 0.95) {
  tail <- (1 - level) / 2
  qbeta(c(tail, 1 - tail), alpha_post, beta_post)
}

pars <- beta_binomial_posterior(alpha0 = 4, beta0 = 16, responders = 14, n = 40)
cri <- equal_tailed_beta_cri(pars["alpha"], pars["beta"])
post_mean <- pars["alpha"] / sum(pars)
prob_above_25 <- 1 - pbeta(0.25, pars["alpha"], pars["beta"])

cat(sprintf("Posterior Beta(%g, %g)\n", pars["alpha"], pars["beta"]))
cat(sprintf("Posterior mean: %.3f\n", post_mean))
cat(sprintf("95%% equal-tailed credible interval: [%.3f, %.3f]\n", cri[1], cri[2]))
cat(sprintf("P(response probability > 0.25 | data): %.3f\n", prob_above_25))

SAS DATA step implementation for exact beta-binomial credible intervals. QUANTILE and CDF provide the same posterior quantiles and threshold probabilities as R and Python for the conjugate model.

data credible_interval;
  alpha0 = 4; beta0 = 16;
  responders = 14; n = 40;
  alpha_post = alpha0 + responders;
  beta_post  = beta0 + (n - responders);
  post_mean  = alpha_post / (alpha_post + beta_post);

  cri_lo = quantile("BETA", 0.025, alpha_post, beta_post);
  cri_hi = quantile("BETA", 0.975, alpha_post, beta_post);
  prob_gt_25 = 1 - cdf("BETA", 0.25, alpha_post, beta_post);

  put "Posterior Beta(" alpha_post comma8. "," beta_post comma8. ")";
  put "Posterior mean: " post_mean 6.3;
  put "95% equal-tailed credible interval: [" cri_lo 6.3 ", " cri_hi 6.3 "]";
  put "P(response probability > 0.25 | data): " prob_gt_25 6.3;
run;

Citations

FOUNDATIONAL / METHODS

APPLIED EXAMPLES

[3]Morey RD, Hoekstra R, Rouder JN, Lee MD, Wagenmakers EJ. The fallacy of placing confidence in confidence intervals. Psychonomic Bulletin & Review. 2016;23(1):103-123.

REPORTING & GUIDANCE