1
$\begingroup$

I've been searching through many books on Bayesian inference but I still can't find anything easy to read and understand the basics of Bayesian hypothesis testing.

I mean I know about prior and posterior distributions. But then I've met the terms like acceptance level, loss function, Bayes factor, least favorable answer, posterior probability, (posterior) quantiles, credible interval, highest posterior density (intervals), ROPE..., and I'm affraid I'm confused about them.

I just don't see what everything (and how) I can use when testing the same hypotheses as in the classical approach (about mean with un/known variance, about ratio of variances, correlation coefficient, probabilities and two-sample tests).

I'm not sure, what everything can be done in Bayesian testing the way I'm used from frequentist testing: p-values, significance level alpha, power and power function, type I and type II errors...

Can you give me an example what to do when I'm given e.g. $Y_1,...,Y_n|\mu,\sigma^2 \sim N (\mu,\sigma^2)$ with a known $\sigma^2$ and I am to test:

$H_0:\mu=\mu_0,\; H_1:\mu\not=\mu_0;$

and

$H_0:\mu\geq\mu_0,\; H_1:\mu<\mu_0.$

I think it would be easier for me to exactly see the steps that need to be taken. The testing process. As it is in classical approach: test statistic and its distibution, select a significance level, critical region, the observed value of the test statistic, decision - reject or not reject the null hypothesis in favor of the alternative.

I'd be so much thankful for any advice :)

$\endgroup$
8
  • $\begingroup$ Do you have a prior for the mean? In your first example, the posterior probability for $H_0$ will vanish unless your prior assigns finite probability mass to $\mu=\mu_0$. $\endgroup$
    – joriki
    Commented Jul 11, 2018 at 10:03
  • $\begingroup$ That's another thing. I've seen you can take various prior distributions and the results will be different. I think conjugate prior distribution might be ok though. $\endgroup$
    – Bee
    Commented Jul 11, 2018 at 10:21
  • $\begingroup$ There's more than one conjugate prior. "Conjugate prior" just means that the prior and posterior belong to the same family of distributions. In this case, a conjugate prior would be a Gaussian distribution for the mean; but you'd still have to specify the parameters of that distribution. Note that, since a Gaussian distribution is continuous, this would lead to zero posterior probability for $\mu$ taking exactly the value $\mu_0$ (and thus posterior probability $1$ for the complementary hypothesis $H_1$ in your first example). $\endgroup$
    – joriki
    Commented Jul 11, 2018 at 11:15
  • $\begingroup$ Yes, I can remember I've read something about testing of point-null hypotheses with absolutely continuous prior distributions. They say you can't use posterior probabilities but you must use credible intervals instead. So then you can take $N(m,s^2)$ as a prior distribution for $\mu$? I'm sorry, I'm lost in the whole Bayesian testing and don't know where to grasp and start. $\endgroup$
    – Bee
    Commented Jul 11, 2018 at 12:07
  • $\begingroup$ For an accessible explanation of frequentist and Bayesian hypothesis testing, side by side, see the article "The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective" at link.springer.com/article/10.3758/s13423-016-1221-4 or preprint at osf.io/ksfyr. For an intro article about Bayesian inference and hypothesis testing, see link.springer.com/article/10.3758/s13423-017-1272-1 or psyarxiv.com/nqfr5. (I am the first author of those articles.) $\endgroup$ Commented Jul 11, 2018 at 14:26

1 Answer 1

0
$\begingroup$

Remember that bayesian inference is done through the posterior distribution then, whenever you're testing hypothesis you've a probability distribution to operate over.

Like so, if you're testing for example $H_0: \mu > k$ you simply calculate that particular probability from your posterior.

Fun fact, you could even perform hypothesis testing just a-priori using just the prior.

$\endgroup$
1
  • $\begingroup$ On the last point, it is wise to consider the prior probability of the null hypothesis: if your null hypothesis has a low prior probability you should not be surprised if it has a low posterior probability, even when the evidence supports it. As an example: suppose your null hypothesis is that a coin is almost fair i.e. has a probability of heads between $0.49$ and $0.51$, while your prior is uniform on $[0,1]$, and you toss it $8$ times getting $4$ heads and $4$ tails, then your posterior probability for the null hypothesis is about $0.0492$ $\endgroup$
    – Henry
    Commented May 20, 2021 at 0:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .