Bayesian Estimation Derivation I am trying to understand Bayesian estimation and I come across this line in my lecture notes:
 θ(Bayesian) = E_θ|x[θ] =  E[π(θ|x)]

So it's meant to reader that the Bayesian estimator is the Conditional Expectation of the sample (x's) which equals the expectation of the posterior (3'rd expression). I understand the derivation up to this point but I dont see how the conditional expectation of theta:
So the middle expression "E_θ|x[θ]" (should look like E underscore θ|x of θ) is:
Integral [θ · π(θ|x) dθ]
and somehow that equals the expectation of the Posterior i.e.
E[π(θ|x)]

Please help! I am hoping it is a simple answer cause there is no explanation between these steps. 
 A: At the begin, you should make sure what is the Bayesian estimator. In chapter 11 of [1], Bayesian estimator is defined as an estimator which minimizes the Bayesian mean square error (Bayesian MSE). Let $\boldsymbol{x}$ be an observed signal, $\theta$ is estimated signal and $\hat{\theta}$ is estimator of $\theta$. With those knowledge, we have 
\begin{align}
\text{Bmse}\  (\hat{\theta})
&=\int (\hat{\theta}-\theta)^2p(\boldsymbol{x},\theta)\text{d}\boldsymbol{x}\text{d}\theta\\
&=\int \left[{\int (\hat{\theta}-\theta)^2p(\boldsymbol{x}|\theta)\text{d}x}\right]p(\theta)\text{d}\theta
\end{align}
Thanks to the non-negativity of $p(\theta)$, the minimum of Bayesian MSE can be touched via minimizing inner integral. To this end, we take partial derivative of inner integral w.r.t $\hat{\theta}$
\begin{align}
\frac{\partial }{\partial \hat{\theta}}\int (\hat{\theta}-\theta)^2p(\boldsymbol{x}|\theta)\text{d}\boldsymbol{x}=-2\int \theta p(\theta|\boldsymbol{x})\text{d}\theta+2\hat{\theta}\int p(\theta|\boldsymbol{x})\text{d}\theta
\end{align}
Let it be 0 yields 
\begin{align}
\hat{\theta}=\int \theta p(\theta|\boldsymbol{x})\text{d}\boldsymbol{x}=\mathbb{E}\left[{\theta|\boldsymbol{x}}\right]
\end{align}
In signal process domain, $p(\theta|\boldsymbol{x})$ is called as posterior distribution.
References 
[1] Sengijpta S K. Fundamentals of statistical signal processing: Estimation theory[J]. 1995. 
A: You first need to take into account that Bayesian Inference derives itself from decision theory.
Second, in Bayesian inference the point estimator $\hat{\theta}$ is the value that minimizes the Expected Loss Function. 
So, for instance, let $\pi(\theta | X_{(n)})$ be the posterior distribution of $\theta$ given the sample $X_{(n)}$ and $\mathcal{L}(\theta, \hat{\theta})$ the Loss function.
So given this two definitions $\hat{\theta}$ will be the value tha minimizes the Expected Loss given by $$\int_{\Theta} \mathcal{L}(\theta, \hat{\theta}) \pi(\theta | X_{(n)})d\theta =\mathbb{E}_{\theta}[L(\theta, \hat{\theta})|X_{(n)}]$$.
Solving this integral for different loss functions yields different point estimators for $\hat{\theta}$, i.e.


*

*$\mathcal{L}= (\theta - \hat{\theta})^2$ yields the expected value/mean of the posterior distribution $\pi$, i.e. $E_{\theta}[\theta|X_{(n)}]$ which is the BMSE in the answer above.

*$\mathcal{L}= |\theta - \hat{\theta}|$ yields the median of the posterior distribution $\pi$.

*$\mathcal{L}= \left\{ \begin{array}{rcl}
1 & \mbox{for}
& |\theta - \hat{\theta}|<\varepsilon \\ 0 & \mbox{for} & |\theta - \hat{\theta}|>\varepsilon
\end{array}\right.$ yields the mode of the posterior posterior $\pi$.


However you can construct any loss function that correctly represents preference in your particular inference decision problem.
I really recommend to you looking at this introductory course in bayesian statistics by Manuel Mendoza to get some real graps on bayesian theory and the axioms underlying Bayesian Inference.
