MLE + Cramér-Rao bound of a discrete random variable Can someone explain how to compute the maximum likelihood estimator, the Fisher information and the Cramér-Rao bound of a discrete random variable please?
I came across this exercise while reviewing for my exam but can't solve it, I had only been dealing with continuous distributions so far.
Let $(X_1,...,X_n)$ be an i.i.d. random sample of size $n$ and distribution $\mathbb{P}_\theta$.
$$\mathbb{P}_\theta(\{0\})=1/4,\hspace{20pt}\mathbb{P}_\theta(\{1\})=3\theta/4,\hspace{20pt}\mathbb{P}_\theta(\{2\})=3(1-\theta)/4$$
$\theta\in[0,1]$
Here's what I've done:
$\mathcal{L}(x_1,...,x_n;\theta)=\left\{\begin{array}{cl}
\dfrac{1}{4}&\text{if } x=0\\
\dfrac{3\theta}{4}&\text{if } x=1\\
\dfrac{3(1-\theta)}{4}&\text{if } x=2\\
\end{array}\right.$
$\ln(\mathcal{L}(x_1,...,x_n;\theta))=\left\{\begin{array}{cl}
\ln(1/4)&\text{if } x=0\\
\ln(3/4)+\ln(\theta)&\text{if } x=1\\
\ln(3/4)+\ln(1-\theta)&\text{if } x=2\\
\end{array}\right.$
$\frac{\partial \ln(\mathcal{L}(x_1,...,x_n;\theta))}{\partial{\theta}}=\left\{\begin{array}{cl}
0&\text{if } x=0\\
1/\theta&\text{if } x=1\\
-1/(1-\theta)&\text{if } x=2\\
\end{array}\right.$
But then I can't figure out the MLE, the Fisher information and the Cramér-Rao bound… Thanks.
 A: As I mentioned in a comment, the likelihood can be written in the following convenient way:
\begin{align}
L(\theta\mid \boldsymbol x)&=\left(\prod_{i: x_i=0}\frac14\right)\left(\prod_{i:x_i=1}\frac{3\theta}4\right)\left(\prod_{i:x_i=2}\frac{3(1-\theta)}4\right)\quad,\,\theta \in [0,1]
\\\\&=\left(\frac14\right)^{n_1}\left(\frac{3\theta}4\right)^{n_2}\left(\frac{3(1-\theta)}4\right)^{n_3}\,,
\end{align}
where $n_1,n_2$ and $n_3$ are respectively the number of observations equal to $0,1$ and $2$ in the sample, such that $n_1+n_2+n_3=n$. These $n_i$'s are of course based on the sample observations, which can be explicitly seen by writing them as sums of indicator variables:
$$n_1(\boldsymbol x)=\sum_{i=1}^n \mathbf1(x_i=0)\,,\,n_2(\boldsymbol x)=\sum_{i=1}^n \mathbf1(x_i=1)\,,\,\text{etc.}$$
In fact, it is clear from the likelihood that $(n_1(\boldsymbol X),n_2(\boldsymbol X),n_3(\boldsymbol X))$ has a multinomial distribution.
This should help you in finding the Fisher information.
A: Or you can compute the second derivative of the log likelihood
$$
\frac{\partial^2 ln L}{\partial \theta^2} = -\frac{3}{4}\left(\frac{N_2}{\theta^2} + \frac{N_3}{(1-\theta)^2}\right)
$$
and take its expectation while noticing that $E(N_2) = 3 n \theta /4$ and $E(N_3) = 3n(1-\theta)/4$. You reteive the desired result since $I_n(\theta) = -E\left(\frac{\partial^2 ln L}{\partial \theta^2}\right) = n I_1(\theta)$ the Fisher information.
