1
$\begingroup$

I have a Bernoulli distribution with the parameter $p$ where the $p$ denotes the probability of $X=1$. Then I have five data points $\{X_i\}$ where $3$ of them are $1$ and $2$ of them are $0$. Thus, I have a likelihood of $\mathcal{L}(\{X_i\}, p)=p^3(1-p)^2$.

Given this, I need to compute the Cramer Rao lower bound of $\hat{p}$ that is estimated through MLE.

My lecture note gives the following formula; however, can't understand how to compute the first part inside the $\mathbb{E}$ - $\frac{\partial^{2}}{\partial \theta_{m} \partial \theta_{k}}$.

Any help or hint how to start?

Proposition from Lecture Note

Let $\hat{\theta}$ be the MLE of $\theta$ using datapoints $\left\{X_{i}\right\}_{i=1}^{n}$. Then we have: $$ \operatorname{Var}(\hat{\theta}) \geq \frac{1}{|\mathcal{I}(\theta)|} $$ Where the Fisher Information $\mathcal{I}(\theta)$ is a matrix defined as: $$ \mathcal{I}(\theta)_{m, k}=\mathbb{E}\left(\frac{\partial^{2}}{\partial \theta_{m} \partial \theta_{k}}-\log \mathcal{L}\left(\left\{X_{i}\right\}, \theta\right)\right) $$

$\endgroup$
1
  • 1
    $\begingroup$ Notice that $\theta_k=p$ is the unknown parameter in your problem, so you're supposed to differentiate twice with respect to $p$. $\endgroup$ Jun 2, 2022 at 16:49

1 Answer 1

1
$\begingroup$

I don't exactly understand whether it is the error in your MathJax, or in the proposition from the lecture notes that causes the confusion. The Fisher Information matrix is supposed to look this way: $$ \mathcal{I}(\theta)_{m, k}=-\mathbb{E}\left(\frac{\partial^{2}}{\partial \theta_{m} \partial \theta_{k}}\log \mathcal{L}\left(\left\{X_{i}\right\}, \theta\right)\right) $$ So to calculate it you have to compute the second derivatives of your log-likelihood function with respect to parameters, just $\hat{p}$ in your case.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .