Bayesian inference exercise I am learning online Bayesian Statistics and I have a test in a couple of days. I have no idea how to solve this exercise, any help will be appreciated. There might be something similar in the quiz...
Statistical decision theory: a decision-theoretic approach to the estimation of an unknown parameter $\theta$ introduces the loss function $L(\theta, a)$ which, loosely speaking, gives the cost of deciding that the parameter has the value $a$, when it is in fact equal to $\theta$. The estimate $a$ can be chosen to minimize the posterior expected loss,
$$E(L(a|y))=  \int L(\theta,a)p(\theta|y)d\theta$$
This optimal choice of $a$ is called a Bayes estimate for the loss function $L$. Show that:
(a) If $L(\theta, a) = (\theta − a)^2$ (squared error loss), then the posterior mean, $E(\theta|y)$, if it exists, is the unique Bayes estimate of $\theta$.
(b) If $L(\theta, a) = |\theta − a|$, then any posterior median of $\theta$ is a Bayes estimate of $\theta$.
(c) If $k_0$ and $k_1$ are non negative numbers, not both zero, and $L(\theta,a)= k_0(\theta−a)$ if $\theta\geq a$,  $k_1(a−\theta)$ if $\theta<a$, then any $k_0$ quantile of the posterior distribution $p(\theta|y)$ is a Bayes estimate of $\theta$.
 A: Here is the derivation for part (a) using slightly different notation. 
If $L(\theta, \hat \theta) = (\theta - \hat \theta)^2$ then we minimize the posterior risk for a given (fixed) value of $z$ as: 
\begin{aligned}
 \arg \min_{\hat \theta} r(\hat \theta | z) & =  \arg \min_{\hat \theta} \int (\theta - \hat \theta)^2 \pi(\theta | z) \ d\theta 
 \\
 \frac{\partial}{\partial \hat \theta} & =  2 \int (\hat \theta - \theta) \pi(\theta | z) \ d\theta = 0
 \\
 \hat \theta \int \pi(\theta | z) \ d\theta & = \int \theta \pi(\theta | z) \ d\theta 
 \\
 \hat \theta & = \int \theta \pi(\theta | z) \ d\theta = E(\theta | z)
\end{aligned}
For the last step recall that $ \int \pi(\theta | z) \ d\theta = 1$ since this is the integral from $- \infty$ to $\infty$ of a probability distribution function, which must be equal to 1. 
Alternatively, we can prove this using a different approach. Since the integral $ \int (\theta - \hat \theta)^2 \pi(\theta | z) \ d\theta $ is over $\theta$ it is effectively the same as the expectation over $\theta$ when treating $\hat \theta$ as a constant. 
$$  \int (\theta - \hat \theta)^2 \pi(\theta | z) \ d\theta = E_\theta[ (\theta - \hat \theta)^2 | z]$$
using expectation algebra this can be re-written in a different form (dropping the $|z$ for notational convenience): 
\begin{aligned}
 E_\theta[ (\theta - \hat \theta)^2] & = E_\theta[ \theta^2 - 2 \theta \hat \theta + \hat \theta^2  ]
 \\
 & = E_\theta[\theta^2] - 2 \hat \theta E_\theta[\theta] + \hat \theta^2
\end{aligned}
We can then differentiate w.r.t. $\hat \theta$ giving: 
\begin{aligned}
 \frac{\partial}{\partial \hat \theta} E_\theta[ (\theta - \hat \theta)^2] & = \frac{\partial}{\partial \hat \theta} E_\theta[\theta^2] - 2 \hat \theta E_\theta[\theta] + \hat \theta^2
 \\
 & = - 2  E_\theta[\theta] + 2 \hat \theta = 0
 \\
 \hat \theta &= E_\theta[\theta]  
\end{aligned}
