# Expectation and random sample

Let $X_1,X_2,...,X_n$ be a random sample of size $n$ from a population $N(\mu, \sigma^2)$ with $\sigma^2=4$. Further assume that $\mu \sim N(0,1)$.

1. Find $E[\mu|X]$ and $E[(\mu - E[\mu|X])^2 |X]$.
2. Find the precision of the normal distribution.
3. Find a 95% credible interval for $\mu$.

So far I understand what to do for 3, and I know that for 2 it is the inverse of the variance. But I cannot figure out 1. I have looked in my book, which is hard to read and I cannot find anything on how to do 1. If I figure out 1, then I can do 2. Please help with 1. Thank you so much!

-
I think I know what to do first. I have to find the posterior distribution. Then based off that I can find the mean and variance. Then I can answer number 1. Sorry for wasting anyones time. I just didn't see what to do until now. – Betty Jul 17 '11 at 7:44
Betty, you can just answer your own question and tick of the solution as the correct one, so as to not leave this question unanswered. – Raskolnikov Jul 17 '11 at 8:22
Hmm -- bad coincidence -- I'd just finished answering it :-) – joriki Jul 17 '11 at 8:26

To find the expectation value and variance of $\mu$ given $X$, you need the conditional probability for $\mu$ given $X$, which is:

$$p(\mu|X)=\frac{p(\mu,X)}{p(X)}\;,$$

with

$$p(X)=\int_{-\infty}^\infty p(\mu,X)\mathrm d\mu\;.$$

For the given distributions, we have

$$p(\mu,X)=\frac1{\sqrt{2\pi}}\mathrm e^{-\mu^2/2}\prod_{i=1}^n\frac1{\sqrt{2\pi}\sigma}\mathrm e^{-(X_i-\mu)^2/(2\sigma^2)}\;.$$

We don't need the constant factors since they drop out when we calculate expectation values. With $S_0:=n$ and $S_k:=\sum\limits_{i=1}^n(X_i)^k$, this becomes

$$p(\mu,X)\propto \mathrm e^{-\mu^2/2} \mathrm e^{-(\mu^2S_0-2\mu S_1+S_2)/(2\sigma^2)}\;,$$

and with $T:=S_0+\sigma^2=n+\sigma^2$ we can rewrite this as

$$\begin{eqnarray} p(\mu,X) &\propto& \mathrm e^{-(\mu^2T-2\mu S_1+S_2)/(2\sigma^2)} \\ &=& \mathrm e^{-T(\mu-S_1/T)^2/(2\sigma^2)}\mathrm e^{-(S_2-S_1^2/T)/(2\sigma^2)} \\ &\propto& \mathrm e^{-T(\mu-S_1/T)^2/(2\sigma^2)}\;, \end{eqnarray}$$

where again we don't need the second factor because it doesn't depend on $\mu$.

So we have a new normal distribution for $\mu$ given $X$, and we can read off the mean $S_1/T$ and the variance $\sigma^2/T$ without having to carry out any integrations.

Analyzing these results is a good exercise in Bayesianology. If we write

$$\frac{S_1}T=\frac{n\bar X-0}{n+\sigma^2}=\frac{n\sigma^{-2}\cdot\bar X-1\cdot0}{n\sigma^{-2}+1}\;,$$

$$\frac{T}{\sigma^2}=\frac{n+\sigma^2}{\sigma^2}=1+n\sigma^{-2}\;,$$

we can see that the precision of the distribution of the mean has increased by the number of measurements times the precision of the measurements, and the new expectation value for the mean is the weighted average of the prior expectation value and the mean of the measurements, weighted by the respective precisions. Thus, things are nicely linear; each measurement adds its precision to the overall precision and its value to the sum of the values, weighted with its precision, and the prior distribution can be treated as if it were an initial measurement.

-
@Didier: What was wrong with $S_0$ defined as the sum of of the zeroth powers? :-) – joriki Jul 17 '11 at 11:40
Of course it is up to you, but I was uneasy about using the notation $x^0$ for (possibly) $x<0$ without notice and I felt that to explicit things could only lead to a better text. – Did Jul 17 '11 at 11:44