Working through this given problem on maximum likelihood estimation (MLE). The density is given as

$$f(x;\theta) = \theta x^{\theta -1} $$ transforming the above equation to MLE, we have

$$L(x;\theta) = \prod_{i=1}^{n} \theta x_i^{\theta - 1} = {\theta}^n \prod_{i=1}^{n} x_i^{\theta - 1}. $$ Taking $\ln$ $$ \begin{align} \ln L(x;\theta) &= \ln{\theta}^n +\ (\theta - 1) \sum_{i=1}^{n} \ln(x_i)\\ \end{align}$$ then we have $$L'(x;\theta) = {n \over \theta} - \sum_{i=1}^{n} \ln(x_i). $$ $$\theta = {n \over \sum_{i=1}^{n} \ln(x_i)}. $$

Is it correct? How can I apply the law of the strong numbers in this case :? Thanks for any help

  • $\begingroup$ no, I resolved it and i dont have idea what should i do next $\endgroup$ – Marta M Jun 28 '17 at 12:23
  • $\begingroup$ I assume from context that $x_i \in (0,1)$ and so the pdf is also defined on $(0,1)$? If so then this seems a bit weird because it forces $\theta<0$... $\endgroup$ – Ian Jun 28 '17 at 12:26
  • $\begingroup$ so when xi is 0 and 1 it will be max? $\endgroup$ – Marta M Jun 28 '17 at 12:27
  • $\begingroup$ $x_i$ are given, not to be chosen in obtaining the MLE. Anyway, my point is the following: suppose the true value of $\theta$ is $2$. I obtain a data set from this distribution; just doing a toy example I get 0.3123 0.3564 0.5277 0.7395 0.7952 0.9026 0.9517 0.9557 0.9785 0.9823. I plug that into your formula for the MLE and I get $\theta^* \approx -2.7759$. What's the deal with that? Why is it so far from the value of $\theta$ I actually used, despite the moderately large data set? $\endgroup$ – Ian Jun 28 '17 at 12:28
  • $\begingroup$ (It also doesn't even make sense for $\theta<0$ in this context because then the pdf is negative...) And now I see the error, the derivative in the log term should have a $+$ sign not a $-$ sign, so you should have $-\frac{n}{\sum_{i=1}^n \ln(x_i)}$ which matches my numerics (with x being the data set I gave above, mle(x,'pdf',@(x,theta) theta*x.^(theta-1),'start',1) returns +2.7759 not -2.7759.) $\endgroup$ – Ian Jun 28 '17 at 12:34

There are some errors in your calculations.

We have \begin{align*} L(x,\theta) &= \prod_1^n \theta x_i^{\theta -1 } \\ \Rightarrow \ln(L(x,\theta)) &= n \ln \theta + \sum_1^n \ln(x_i^{\theta - 1}) \\ &= n \ln \theta + (\theta - 1) \sum_1^n \ln(x_i) \\ \end{align*} Taking the derivative with respect to $\theta$ yields \begin{align*} \frac{d[\ln(L(x,\theta))]}{d\theta} = \frac{n}{\theta} + \sum_1^n \ln(x_i) \end{align*} Setting the derivative equal to 0 and solving for $\theta$ gives the MLE, \begin{align*} \hat{\theta} = \frac{-n}{\sum_1^n \ln(x_i)} \end{align*} You must have forgotten to specify that $x_i \in (0,1)$, otherwise we could have a negative parameter estimate, which gives a negative pdf (this is impossible).


Not the answer you're looking for? Browse other questions tagged or ask your own question.