12
$\begingroup$

Let $X_1,X_2,\ldots,X_n$ be i.i.d $\text{P}(\lambda)$ random variables where $\lambda(>0)$ is unknown. Define $$\bar X=\frac{1}{n}\sum_{i=1}^n X_i\qquad,\qquad S^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X)^2$$ as the sample mean and sample variance respectively.

Since $\sum_{i=1}^n X_i$ and hence $\bar X$ is a complete sufficient statistic for $\lambda$ such that $E(\bar X)=\lambda$, $\bar X$ is the uniformly minimum variance unbiased estimator (UMVUE) of $\lambda$ by the Lehmann-Scheffe theorem. Again, $E(S^2)=\lambda$, so that $E(S^2\mid \bar X)$ is also the UMVUE of $\lambda$. As UMVUE is unique whenever it exists, it must be that $$E(S^2\mid \bar X)=\bar X$$

The question is:

How can I directly show that $E(S^2\mid \bar X)=\bar X$ ?

I don't see how to proceed from

\begin{align} E(S^2\mid \bar X=t)&=E\left[\frac{1}{n-1}\sum_{i=1}^n(X_i-t)^2\mid \bar X=t\right] \\&=E\left[\frac{1}{n-1}\left(\sum_{i=1}^nX_i^2-nt^2\right)\mid \bar X=t\right] \\&=E\left[\frac{1}{n-1}\sum_{i=1}^nX_i^2\mid \bar X=t\right]-E\left[\frac{nt^2}{n-1}\mid \bar X=t\right] \\&=\frac{1}{n-1}\sum_{i=1}^nE\left(X_i^2\mid \bar X=t\right)-\frac{n}{n-1}E(\bar X^2\mid \bar X=t)\tag{1} \end{align}

Any hint would be great.


As correctly pointed out by Mike Earnest, the conditional distribution of $(X_1,X_2,\cdots,X_n)\mid \bar X$ is multinomial. That is, for a natural number $k$,

$$P\left(X_1=x_1,\cdots,X_n=x_n\mid \bar X=\frac{k}{n}\right)=\frac{k!}{x_1!\,x_2!\cdots x_n!}\left(\frac{1}{n}\right)^{x_1}\left(\frac{1}{n}\right)^{x_2}\cdots\left(\frac{1}{n}\right)^{x_n}\mathbf1_{x_i\in A}$$

, where $$A=\left\{(x_1,\cdots,x_n)\in\{0,1,\cdots,k\}: \sum_{i=1}^nx_i=k\right\}$$

From this, we have for each $i$, $$V\left(X_i\mid \bar X=t\right)=t\left(1-\frac{1}{n}\right)\qquad,\qquad E\left(X_i\mid \bar X=t\right)=t$$

And for all $i\ne j$, $$E\left(X_iX_j\mid \bar X=t\right)=t\left(t-\frac{1}{n}\right)$$

So,

\begin{align} E\left(X_i^2\mid \bar X=t\right)&=V\left(X_i\mid \bar X=t\right)+\left[E\left(X_i\mid \bar X=t\right)\right]^2 \\&=\frac{t}{n}(n+nt-1) \end{align}

Also, as expected,

\begin{align} E\left(\bar X^2\mid \bar X=t\right)&=E\left[\frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^nX_iX_j\mid \bar X=t\right] \\&=\frac{1}{n}E\left(X_1^2\mid \bar X=t\right)+\frac{1}{n^2}\sum_{i\ne j}E\left[X_iX_j\mid \bar X=t\right] \\&=\frac{1}{n}\cdot\frac{t}{n}(n-1+nt)+\frac{2}{n^2}\binom{n}{2}t\left(t-\frac{1}{n}\right) \\&=t^2 \end{align}

So from $(1)$ I finally get,

\begin{align} E(S^2\mid \bar X=t)&=\frac{n}{n-1}\cdot\frac{t}{n}(n+nt-1)-\frac{n}{n-1}\cdot t^2 \\&=t \end{align}

Hence proved.

(Thanks to Mike Earnest in particular.)

$\endgroup$
9
  • 2
    $\begingroup$ Hint: You can show that whenever $X_i\sim \text{Poi}(\lambda)$ are iid, then $$ P(X_1=x_1,\dots,X_n=x_n|X_1+\dots+X_n=k)=\frac1{n^k}\binom{k}{x_1,x_2,\dots,x_n} $$ That is, conditioned on $\overline X=k/n$, the vector $(X_1,\dots,X_n)$ has a multinomial distribution. $\endgroup$ Commented Aug 13, 2018 at 20:53
  • $\begingroup$ @MikeEarnest Yes I can show that. Let me see how that helps here. $\endgroup$ Commented Aug 13, 2018 at 21:14
  • 2
    $\begingroup$ You have some mistakes in your edit, probably stemming from the fact that your $k$ is Wikipedia’s $n$. For example, $EX_i=k/n$, not $1$. I think fixing that should make you good to go! $\endgroup$ Commented Aug 14, 2018 at 12:49
  • 1
    $\begingroup$ @SextusEmpiricus $S^2$ is unbiased for $\lambda$ and $\bar X$ is complete sufficient. So that follows by Lehmann-Scheffé. $\endgroup$ Commented Apr 15, 2021 at 7:43
  • 1
    $\begingroup$ @Stubborn I got it. LS: $E(Y|X)$ is UMVUE if $Y$ is unbiased and if $X$ is complete sufficient. $\endgroup$ Commented Apr 15, 2021 at 12:49

1 Answer 1

5
$\begingroup$

$\newcommand{\e}{\operatorname{E}}$ \begin{align} & \Pr(X_1=x_1 \mid \overline X = x/n) = \Pr(X_1=x_1\mid X_1+\cdots+X_n = x) \\[10pt] = {} & \frac{\Pr(X_1=x_1\ \&\ X_1+\cdots+X_n = x)}{\Pr(X_1+\cdots+X_n = x)} \\[10pt] = {} & \frac{\Pr(X_1=x_1\ \&\ X_2+\cdots+X_n = x-x_1)}{\Pr(X_1+\cdots+X_n = x)} \\[10pt] = {} & \frac{\dfrac{\lambda^{x_1} e^{-\lambda}}{(x-x_1)!} \cdot \dfrac{((n-1)\lambda)^{x-x_1} e^{-((n-1)\lambda)}} {x_1!}}{\left( \dfrac{(n\lambda)^x e^{-n\lambda}} {x!} \right)} = \binom x {x_1} \left( \frac 1 n \right)^x \left( 1 - \frac 1 n \right)^{x-x_1} \end{align} In other words, $$ X_1\mid \overline X \sim \operatorname{Binomial} \left(n\overline X, \frac 1 n \right). $$ Therefore $$ \operatorname E\left((X_1-\overline X)^2 \mid \overline X\right) = \overline X\left( 1 - \frac 1 n \right). $$ Consequently $$ \operatorname E\left( (X_1-\overline X)^2 + \cdots + (X_n-\overline X)^2 \mid \overline X\right) = (n-1)\overline X. $$

$\endgroup$
1
  • $\begingroup$ Yes, a valid option was to find the conditional distribution. It should have occurred to me that $X_i\mid \bar X$ is binomial right at the start, since it is well known that $X_i\mid \sum X_i$ is binomial. $\endgroup$ Commented Aug 16, 2018 at 19:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .