0
$\begingroup$

Consider the following linear regression model: $Y=\beta_0 + \beta_1 x_i + error $, where $y_i,...,y_n$ are normally distributed and $error \sim N(0,\sigma^2)$. I am trying to find the MLEs for $\beta_0$ and $\beta_1$.

The likelihood function is:

$$ L(\beta_0,\beta_1,\sigma^2) = \prod_{i=1}^{n} \frac{e^{-\frac{(y_i - (\beta_ + \beta_1 x_i))^2}{2\sigma^2}}}{\sqrt{2\pi\sigma^2}} $$

The log-likelihood function is:

$$ l(\beta_0,\beta_1,\sigma^2) = \sum_{i=1}^{n} {-\frac{(y_i - (\beta_0 + \beta_1 x_i))^2}{2\sigma^2}} - \frac{1}{2}log(2\pi\sigma^2) $$

Next, I differentiate to find the MLEs:

$$ \frac{\partial}{\partial\beta_0} l(\beta_0,\beta_1,\sigma^2) = 0 \rightarrow n\bar{y} - n\beta_0 - n\beta_1\bar{x} = 0 \rightarrow \hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x} $$

$$ \frac{\partial}{\partial\beta_1} l(\beta_0,\beta_1,\sigma^2) = 0 \rightarrow \sum_{i=1}^{n} y_i x_i - n \beta_0\bar{x} - \beta_1\sum_{i=1}^{n} x_i^2 = 0 $$

Now plugging $\hat{\beta_{0}}$ from equation 1 into equation 2:

$$ \sum_{i=1}^{n} y_i x_i - n\big(\bar{y} - \hat{\beta_1}\bar{x}\big)\bar{x} - \hat{\beta_1}\sum_{i=1}^{n} x_i^2 = 0 \\ \sum_{i=1}^{n} y_i x_i - n\bar{y}\bar{x} = \hat{\beta_1}\sum_{i=1}^{n} x_i^2 - n\hat{\beta_1}\bar{x_i}^2 \\ \hat{\beta_1} = \frac{\sum_{i=1}^{n} y_i x_i - n\bar{y}\bar{x}}{\sum_{i=1}^{n} x_i^2 - n\bar{x_i}^2} $$

My answer for $\hat{\beta_0}$ appears correct. However, every textbook that I have consulted shows that $\hat{b_1} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y}) }{\sum_{i=1}^{n} (x_i - \bar{x})^2} $. Where have I gone wrong?

$\endgroup$
1
$\begingroup$

We have $$\begin{eqnarray}\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y) &=&\sum_{i=1}^n x_iy_i- \bar x\sum_{i=1}^ny_i - \bar{y}\sum_{i=1}^nx_i + \bar x\bar y \sum_{i=1}^n (1) \\&=& \sum_{i=1}^nx_iy_i-n\bar x\bar y - n \bar x \bar y + n\bar x \bar y \\ &=&\sum_{i=1}^n x_i y_i - n \bar x \bar y\end{eqnarray}$$

so your numerator's right. Maybe your denominator is too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.