2
$\begingroup$

I am self studying Bayes Decision theory from these lecture notes page 30 / 31 and there is a step a struggle to understand mathematically

Background context

Given Bayes risk defined as:

$$ r_B(\pi, \hat \theta) = \int_{\Theta} R(\theta, \hat \theta) \ \pi(\theta) \ d \theta$$

  • Prior distribution $\theta \sim \pi(\theta)$
  • Distribution of data given $\theta$ $z | \theta \sim f(z | \theta)$
  • Marginal distribution of $z$: $m(z) = \int f(z | \theta) \pi (\theta) \ d\theta$
  • Posterior distribution of $\theta$ $\pi(\theta | z) = \frac{f(z | \theta) \pi(\theta)}{m(z)}$
  • Frequentist risk $R$: $R(\theta, \hat \theta) = E[L(\theta, \hat \theta)| \theta]$

We can express Bayes Risk in terms of the posterior risk, a function of $z$

$$ r_B(\pi, \hat \theta) = \int_{\mathcal{Z}} r(\hat \theta | z) m(z) dz$$

Which means that the Bayes decision rule can be obtained as

\begin{aligned} r_b(\pi, \hat \theta_\pi^{Bayes}) &= \inf_{\hat \theta} r_b(\pi, \hat \theta) \\ &= \inf_{\hat \theta} \int_{\mathcal{Z}} r(\hat \theta | z) m(z) dz \end{aligned}

Step I don't grasp

The above theorem implies that the Bayes rule can be obtained by taking the Bayes action for each particular $z$. For each fixed $z$ we choose $\hat \theta(z)$ to minimize the posterior risk $r(\hat \theta | z)$.

$$ \arg \min_{\hat \theta} \ r(\hat \theta | z) = \int L(\theta, \hat \theta(z)) \pi(\theta | z) \ d\theta $$

This guarantees us to minimize the integrand at every $z$ and hence minimize Bayes risk.

How do we get from the inf to the argmin and why does one minimize the other ? I feel this is a stupid question but I am missing something... thanks !

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.