1
$\begingroup$

Consider the following linear regression model: $Y=\beta_0 + \beta_1 x_i + error $, where $y_i,...,y_n$ are normally distributed and $error \sim N(0,\sigma^2)$. I am trying to find the MLEs for $\beta_0$ and $\beta_1$.

The likelihood function is:

$$ L(\beta_0,\beta_1,\sigma^2) = \prod_{i=1}^{n} \frac{e^{-\frac{(y_i - (\beta_ + \beta_1 x_i))^2}{2\sigma^2}}}{\sqrt{2\pi\sigma^2}} $$

The log-likelihood function is:

$$ l(\beta_0,\beta_1,\sigma^2) = \sum_{i=1}^{n} {-\frac{(y_i - (\beta_0 + \beta_1 x_i))^2}{2\sigma^2}} - \frac{1}{2}log(2\pi\sigma^2) $$

Next, I differentiate to find the MLEs:

$$ \frac{\partial}{\partial\beta_0} l(\beta_0,\beta_1,\sigma^2) = 0 \rightarrow n\bar{y} - n\beta_0 - n\beta_1\bar{x} = 0 \rightarrow \hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x} $$

$$ \frac{\partial}{\partial\beta_1} l(\beta_0,\beta_1,\sigma^2) = 0 \rightarrow \sum_{i=1}^{n} y_i x_i - n \beta_0\bar{x} - \beta_1\sum_{i=1}^{n} x_i^2 = 0 $$

Now plugging $\hat{\beta_{0}}$ from equation 1 into equation 2:

$$ \sum_{i=1}^{n} y_i x_i - n\big(\bar{y} - \hat{\beta_1}\bar{x}\big)\bar{x} - \hat{\beta_1}\sum_{i=1}^{n} x_i^2 = 0 \\ \sum_{i=1}^{n} y_i x_i - n\bar{y}\bar{x} = \hat{\beta_1}\sum_{i=1}^{n} x_i^2 - n\hat{\beta_1}\bar{x_i}^2 \\ \hat{\beta_1} = \frac{\sum_{i=1}^{n} y_i x_i - n\bar{y}\bar{x}}{\sum_{i=1}^{n} x_i^2 - n\bar{x_i}^2} $$

My answer for $\hat{\beta_0}$ appears correct. However, every textbook that I have consulted shows that $\hat{b_1} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y}) }{\sum_{i=1}^{n} (x_i - \bar{x})^2} $. Where have I gone wrong?

$\endgroup$

1 Answer 1

2
$\begingroup$

We have $$\begin{eqnarray}\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y) &=&\sum_{i=1}^n x_iy_i- \bar x\sum_{i=1}^ny_i - \bar{y}\sum_{i=1}^nx_i + \bar x\bar y \sum_{i=1}^n (1) \\&=& \sum_{i=1}^nx_iy_i-n\bar x\bar y - n \bar x \bar y + n\bar x \bar y \\ &=&\sum_{i=1}^n x_i y_i - n \bar x \bar y\end{eqnarray}$$

so your numerator's right. Maybe your denominator is too.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .