Let $Y_i=\alpha_0+\beta_0 X_i + \epsilon_0$, where $\epsilon_i \sim N(0, \sigma_0^2)$ and $X_i \sim N(\mu_x,\tau_0^2)$ are independent.

The data $(X_i, Y_i)$ are generated from $Y_i=\alpha_0+\beta_0 X_i + \epsilon_0$.

I have found the maximum likelihood estimator for each parameter by using $$L_n({X_i, Y_i};\, \alpha, \beta, \mu_x, \sigma^2, \tau^2) = \prod_{i=1}^n f(X_i, Y_i)=\prod_{i=1}^n f_x(X_i)f_{.|X_i}(Y_i),$$ differentiating it with respect to the parameter, setting it equal to zero, and solving for the parameter.

For example, I get that $\hat{\alpha}_{MLE}=\bar{Y_n}-\beta \bar{X_n}$.

But how do I show that $\hat{\alpha}_{MLE}=\bar{Y_n}-\beta \bar{X_n}$ converges to its true value?

I do not know how to find the true value of $\hat{\alpha}_{MLE}$ or how to begin showing convergence.

Thank you.

  • $\begingroup$ I have tried to edit your question for readability, but the notation at the end of the displayed equation (not changed) is unclear to me. Please re-edit as required. $\endgroup$ – BruceET Dec 3 '17 at 17:19

You have by the law of total expectation that \begin{equation} E[Y_i] = E[E[Y_i | X_i]] = E[\alpha_0 + \beta_0\mu_x + \epsilon_i] = \alpha_0 + \beta_0\mu_x \end{equation}

You also have by the law of large numbers that $\bar{X_n} \overset{P}{\rightarrow} \mu_x$. By Slutsky's theorem, we have that:

\begin{equation} \bar{Y_n} -\beta_0\bar{X_n} \overset{P}{\rightarrow} Y - \beta_0\mu_x \end{equation}

Where $Y$ is the limiting distribution of $\bar{Y_n}$. Again using the law of large numbers we have that $\bar{Y_n} \overset{P}{\rightarrow} Y = E[Y_i] = \alpha_0+\beta_0\mu_x$.

Plugging these into the previous expression we have that \begin{equation} \bar{Y_n} - \beta_0\bar{X_n} \overset{P}{\rightarrow} \alpha_0 +\beta_0\mu_x - \beta_0\mu_x = \alpha_0 \end{equation}

  • $\begingroup$ Thank you very much for your clear answer. Do you know if it is reasonable that $\hat\sigma_{MLE}\overset{P}{\rightarrow} 0$ and $\hat\tau_{MLE}\overset{P}{\rightarrow} 0$? $\endgroup$ – Silvia Rossi Dec 4 '17 at 5:43
  • $\begingroup$ No, the MLE for $\sigma^2$ is $\hat{\sigma}^2=\frac{1}{n}\sum_i(X_i - \bar{X})^2$, which has sampling distribution: \begin{equation} \frac{n\hat{\sigma}^2}{\sigma^2} \sim \chi^2_{n-1} \end{equation} Which implies that the expectation of $E[\hat{\sigma}^2] = \frac{\sigma (n-1)}{n}$. By the law of large numbers $\hat{\sigma}^2 \overset{P}{\rightarrow} \sigma^2$. $\endgroup$ – Ryan Warnick Dec 4 '17 at 5:56
  • $\begingroup$ Ah okay, something definitely went wrong for me, thank you again :) $\endgroup$ – Silvia Rossi Dec 4 '17 at 6:01
  • $\begingroup$ No problem, good luck. $\endgroup$ – Ryan Warnick Dec 4 '17 at 6:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.