Let $X_1, X_2,X_3\sim\rm{ Bernoulli}(\theta)$. Show that $I_{X_1+X_2>X_3}$ is an unbiased estimator of $h(\theta)$ and find UMVUE of $h(\theta)$

Let $$X_1, X_2,X_3\sim\rm{ Bernoulli}(\theta)$$. Show that $$I_{X_1+X_2>X_3}$$ is an unbiased estimator of $$h(\theta)=P_{\theta}(X_1+X_2>X_3)$$ and find UMVUE of $$h(\theta)$$.

Assuming $$A = (X_1+X_2 > X_3)$$, is it correct that $$E[I_A]= P(A)$$ in this situation?

If this is correct, then I know that for the estimator to be unbiased, $$E[I_A] = P(A) = h(\theta)$$, which checks out.

I also know that $$I_A$$ is the best unbiased estimator of $$h(\theta)$$ if it attains the Cramer-Rao lower bound:

$$\operatorname{Var}\left(I_A\right) \geq \frac{\left(\frac{d}{d\theta}E[I_A]\right)^2}{E\left[\left(\frac{d}{d\theta} \ln\left(f(X\mid\theta)\right)\right)^2\right]}$$

When I try solving for $$\operatorname{Var}\left(I_A\right)$$:

\begin{align} \operatorname{Var}\left(I_A\right) &= E[(I_A - E[I_A])^2] \\ &= E[(I_A - h(\theta))^2] \\ \end{align}

I get stuck because I don't know how to simplify further (i.e. I don't know how to evaluate $$h(\theta)$$).

Trying to solve for the left-hand side, I am also stuck because I don't know the joint pmf $$f(X\mid\theta)$$ that gives rise to $$h(\theta)$$ when integrated.

Edit:

Because $$I_A$$ is not the UMVUE, I need to find the best unbiased estimator of $$h(\theta)$$. Toward this end, I figured out that the joint pmf is $$f(X\mid\theta) = \theta^{(X_1+X_2+X_3)}(1-\theta)^{(3-(X_1+X_2+X_3))}$$ and $$h(\theta) = \theta^2 + 2 \theta (1-\theta)^2$$ but how do I use this information to obtain an estimator $$\delta$$ such that $$E[\delta] = h(\theta)$$?

• You must have $X_1,X_2,X_3$ independent of each other, which gives you joint pmf as well as $P(A)$. And variance of $I_A$ is just $P(A)(1-P(A))$. Nov 17, 2019 at 20:38
• Also, the UMVU estimator should be based on a complete sufficient statistic by en.wikipedia.org/wiki/Lehmann%E2%80%93Scheff%C3%A9_theorem. So $I_A$ does not seem to be the unbiased estimator with minimum variance. Nov 17, 2019 at 21:04
• Thank you for your help! Looks like I misread my original problem, which only asked me to show that $I_A$ is unbiased, then find the best unbiased estimator of $h(\theta)$
– N.B.
Nov 17, 2019 at 21:58
• @StubbornAtom : They don't need to be independent in order to have the expected value of that indicator equal to the probability of that event. However, before you can say that indicator is an unbiased estimator of $\theta,$ you need to say that that probability is a function of $\theta.$ One way to do that involves saying that $X_1,X_2,X_3$ are independent. But if, for example, $X_1,X_2$ were independent and $X_3$ were the mod-$2$ sum of $X_1$ and $X_2,$ then that indicator would still be an unbiased estimator of that probability. Nov 18, 2019 at 2:16
• @MichaelHardy The point was that OP should not overlook the details of the actual question. He/she must have been assigned this exercise with the usual independence assumption. If not, then that should be mentioned clearly. Nov 18, 2019 at 13:52

2 Answers

The expected value of an indicator function is the probability that it is equal to $$1.$$ Thus it is $$\Pr(X_1+X_2=1 \text{ or } X_3=0).$$ That is enough to show that the indicator function is an unbiased estimator of that probability.

Now assuming $$X_1,X_2,X_3$$ are independent, one can show that the conditional distribution of $$(X_1,X_2,X_3)$$ given $$X_1+X_2+X_3$$ does not depend on $$\theta.$$ Thus the conditional expected value of the unbiased estimator given $$X_1+X_2+X_3$$ should give you an improved estimator.

We have \begin{align} & \Pr(X_1+X_2>X_3\mid X_1+X_2+X_3=x) \\[8pt] = {} & \begin{cases} 1 & \text{if } x = 3 \\ 1/3 & \text{if } x = 2 \\ 2/3 & \text{if } x = 1 \\ 0 & \text{if } x = 0 \end{cases} \tag 1 \end{align} Thus the function of $$x$$ defined by line $$(1)$$ above, when evaluated at $$X_1+X_2+X_3,$$ gives you an improved estimator. And here I am actually surprised by the non-monotone nature of this function.

This gives you the best unbiased estimator if the statistic $$X_1+X_2+X_3$$ is complete. We have \begin{align} & \operatorname E(g(X_1+X_2+X_3)) \\[10pt] = {} & \theta^3 g(3) + 3\theta^2(1-\theta)g(2) \\[3pt] & {} + 3\theta(1-\theta)^2 g(1) + (1-\theta)^3 g(0) \\[10pt] = {} & \big( g(3) -3g(2) + 3g(1) - g(0) \big)\theta^3 \\[3pt] & {} + \big( 3g(2) - 6 g(1) + 3g(0) \big) \theta^2 \\[3pt] & {} + \big( 3g(1) - 3g(0) \big) \theta \\[3pt] & {} + \big( g(0) \big). \end{align} This can remain equal to $$0$$ as $$\theta$$ changes only if \begin{align} g(3) -3g(2) + 3g(1) - g(0) & = 0, \\[4pt] 3g(2) - 6 g(1) + 3g(0) & = 0, \\[4pt] 3g(1) - 3g(0) & = 0, \\[4pt] g(0) & = 0. \end{align} This holds only if $$g(3)=g(2)=g(1)=g(0)=0.$$

Thus we have completeness.

Finally, the Lehmann–Scheffé theorem says we have the UMVUE.

Assuming independence of $$X_1,X_2,X_3$$, the statistic $$T=X_1+X_2+X_3$$ is complete sufficient for $$\theta$$ since the parent distribution is a member of the exponential family.

An unbiased estimator of $$h(\theta)$$ based on $$T$$ is the UMVUE of $$h(\theta)$$ by Lehmann-Scheffe theorem.

Now as you have found, $$h(\theta)=2\color{blue}{\theta^3}-3\color{green}{\theta^2}+2\color{red}{\theta}$$.

Using the form of the UMVUE of $$\theta^k$$ for $$k\in\mathbb N$$, the UMVUE of $$h(\theta)$$ is clearly

$$g(T)=2\cdot\color{blue}{\frac{T(T-1)(T-2)}{3\cdot2\cdot1}}-3\cdot\color{green}{\frac{T(T-1)}{3\cdot2}}+2\cdot\color{red}{\frac{T}{3}}$$

That is, $$g(T)=\frac{T^3}{3}-\frac{3T^2}{2}+\frac{11T}{6}$$