# Convergence of empirical distribution

Let $Z_1 , Z_2 , \ldots$ be iid with $E(|Z_i|) < \infty$. Let $\theta$ be independent of the $Z_i$ with $E(|\theta|) < \infty$. Let $Y_i = \theta + Z_i$. I am trying to show that the conditional expectations $E(\theta\mid Y_1 ,\ldots, Y_n)$ converge a.s. to $\theta$. I know that $E(\theta\mid Y_1 ,\ldots , Y_n) \rightarrow E(\theta \mid \sigma(Y_1, Y_2 , \ldots))$ a.s. as $n\rightarrow \infty$, so it will suffice to show that $E(\theta \mid \sigma(Y_1, Y_2 , \ldots)) = \theta$. (here $\sigma(Y_1 , Y_2 , \ldots)$ is the $\sigma$ algebra generated by the $Y_i$) However, I do not see a way to do this. Any suggestions?

By the Strong Law of Large Numbers, there's a set $\Omega$ of measure 1 on which $$\lim_{n\to\infty}\frac{Z_1+\cdots+Z_n}{n} = \mathbb{E}(Z_1).$$

So, on this set, we have

$$\theta+ \lim_{n\to\infty}\frac{Z_1+\cdots+Z_n}{n} = \theta + \mathbb{E}(Z_1).$$

That is,

$$\lim_{n\to\infty}\frac{(\theta+Z_1)+\cdots+(\theta+Z_n)}{n} = \theta + \mathbb{E}(Z_1).$$

I.e.

$$\lim_{n\to\infty}\frac{Y_1+\cdots+Y_n}{n} = \theta + \mathbb{E}(Z_1).$$

Since $\theta$ is (a.s.) equal to a $\sigma(Y_1,Y_2,\cdots)$-measurable random variable,

$$\mathbb{E}(\theta|\sigma(Y_1,Y_2,\cdots)) = \theta \text{ a.s.}$$

so we're done.

• Did you need to use the $\theta$ independence here? I am guessing you implicitly use it in the final one to get $Y_{i}\sim N(\theta,1)$ are iid conditionally on $\theta$ and then apply SLLN on them. – OOESCoupling Feb 22 at 21:18
• @OOESCoupling, no. The argument applies the SLLN once to $Z_1,Z_2,\cdots$. – Ben Derrett Feb 22 at 22:00