# Almost sure convergence of posterior distribution to parameter

I'm trying to solve an exercise form Rick Durrett's book on probability, following a section on almost sure convergence of martingales:

Let $Z_1,Z_2,...$ be i.i.d standard normal random variables, let $\theta$ be an independent random variable with finite mean, and let $Y_i:=Z_i+\theta$. $\;$ Show that $\mathbb{E}[\theta \vert Y_1,...,Y_n] \overset{n\rightarrow \infty}{\rightarrow}\theta$ almost surely.

I know that $\mathbb{E}\Big[ \frac{1}{n} \sum_{i=1}^n Y_i \Big \vert Y_1,...,Y_n \Big]= \frac{1}{n} \sum_{i=1}^nY_i$, and that by the strong law of large numbers we know that $\frac{1}{n} \sum_{i=1}^nY_i \overset{n\rightarrow \infty}{\rightarrow}\theta$ almost surely. I'm unsure as to how best to proceed from here, or in fact how to use the independence of $\theta$. $\;$ I would appreciate any help or hints.

• Isn't it a direct consequence of theorem 5.5.7: suppose $\mathcal{F}_n\uparrow\mathcal{F}_\infty$, i.e., $\mathcal{F}_n$ is an increasing sequence of $\sigma$-algebras and $\mathcal{F}_\infty=\sigma(\cup_n\mathcal{F}_n)$ then $E(X|\mathcal{F}_n)\to E(X|\mathcal{F}_\infty)$ almost surely? Feb 1 '18 at 20:17

Define: \begin{align} \F_n=\sigma(Y_1,...,Y_n) \end{align} As noted in the comments we have: \begin{align} \E[\theta|\F_n]\to \E[\theta|\F_\infty] \ \ \ \ \ \text{ a. s.} \end{align} But unfortunately that is not enough. We don't know whether $\E[\theta|\F_\infty]=\theta$ holds. That is what we will show. Before showing that define $$X_n:= \frac{1}{n}\sum_{k=1}^n Y_k$$ Clearly $X_n$ is $\F_\infty$-measurable.
Conditioning on $\theta$ we know that $Y_i\sim \mathcal N(\theta,1)$ i.i.d. So: \begin{align} \E[X_n|\theta]=\theta \ \ \ \ \ \text{ and } \ \ \ \ \ \E[(X_n-\theta)^2|\theta]=\frac{1}{n} \end{align} Taking the expectation of both sides of the latter: \begin{align} \E[(X_n-\theta)^2]=\frac{1}{n} \end{align} That implies $X_n\to \theta$ in $L^2$. One knows that there is a subsequence $X_{n_k}$ that converges to $\theta$ a. s.. There is $\hat\theta$ which is $\F_\infty$-measurable and $\theta=\hat \theta$ a. s. (why?). Hence $$\E[\theta|\F_\infty] = \E[\hat\theta |\F_\infty] =\hat\theta$$ Conclusion: \begin{align} \E[\theta|Y_1,...,Y_n]\to \theta \ \ \ \ \ \text{ a. s. } \end{align}
• This may be something trivial which I am not seeing, but why is $\mathbb{E}[Y_i\vert \theta] \sim \mathcal{N}(0,1)$ and i.i.d? Feb 2 '18 at 6:33
• @Keen-ameteur I did not claim $E[Y_i|\theta]$ are iid normally distributed, but $Y_i|\theta =t\sim N(t, 1)$. Okay, $\theta$ is a random variable. If I would now it's value, say $\theta=t$ then $Y_i=Z_i+\theta=Z_i+t$, right? Since $Z_i$ are iid standard normally distributed, so are $Y_i$ iid normally distributed with mean $t$. Actually I should say $Y_i|\theta=t\sim N(t, 1)$ iid. Is it clear? Feb 2 '18 at 10:01
• @Keen-ameteur $E[Y_i|\theta] =\theta$. So it's not normally distributed as long as $\theta$ is not normally distributed. The statement I have made is quite different as you could see in my previous comment. Feb 2 '18 at 10:08