0
$\begingroup$

I am given the model $Y_i=\alpha_0+\beta_0 X_i+\epsilon_0$, where $i=1,2,...,n$, $X_i$ are fixed numbers and $\epsilon \sim N(0, \sigma^2).$ I am also given that $\sigma^2$ and the parameters $(\alpha_0,\beta_0)$ for $E(Y_i)=\alpha_0+\beta_0 X_i$ are unknown.

I have to estimate $(\alpha_0,\beta_0)$ using $(\alpha^*,\beta^*)$ which are found by minimizing $\sum_{i=1}^n (Y_i-\alpha-\beta X_i)^2.$

To find $\alpha^*$, I set $\frac{d}{d\alpha}\sum_{i=1}^n (Y_i-\alpha-\beta X_i)^2=0$ and got that $\alpha^*=\widehat{Y}-\beta^* \widehat{X}$, where $\widehat{Y}=\frac{Y_1+Y_2+...+Y_n}{n}$ and $\widehat{X}=\frac{X_1+X_2+...+X_n}{n}$.

However I am having a hard time finding $\beta^*$. Here is my process and where I get stuck: $\frac{d}{d\beta}\sum_{i=1}^n (Y_i-\alpha-\beta X_i)^2=0$

$\sum_{i=1}^n \frac{d}{d\beta}(Y_i-\alpha-\beta X_i)^2=0$

$\sum_{i=1}^n X_i(Y_i-\alpha-\beta X_i)=0$

$\sum_{i=1}^n (Y_i X_i-\alpha X_i-\beta X_i^2)=0$

$\sum_{i=1}^n (Y_i X_i-(\widehat{Y}-\beta \widehat{X}) X_i-\beta X_i^2)=0$

$\sum_{i=1}^n (Y_i X_i-\widehat{Y} X_i+\beta \widehat{X} X_i-\beta X_i^2)=0$

$\sum_{i=1}^n (Y_i X_i-\widehat{Y} X_i+\beta (\widehat{X} X_i-X_i^2))=0$

$\sum_{i=1}^n (Y_i X_i-\widehat{Y} X_i+\beta (\widehat{X} X_i-X_i^2))=0$

$\sum_{i=1}^n \beta (\widehat{X} X_i-X_i^2)=\sum_{i=1}^n(\widehat{Y} X_i-Y_i X_i)$

$\beta^* =\frac{\sum_{i=1}^n(\widehat{Y} X_i-Y_i X_i)}{\sum_{i=1}^n(\widehat{X} X_i-X_i^2)}$

However, $\beta^*$ should equal $\frac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}$.

Could someone please explain what went wrong? Thank you.

Ps I am sorry for any mistakes in typing the given information, I am not too familiar with the topic.

$\endgroup$
1
$\begingroup$

You are on the right track: $$\color{blue}{\beta^* =\frac{\sum_{i=1}^n(\widehat{Y} X_i-Y_i X_i)}{\sum_{i=1}^n(\widehat{X} X_i-X_i^2)}}=\frac{\widehat{Y}\sum X_i-\sum Y_i X_i}{-\widehat{X}\sum X_i+2\widehat{X}\sum X_i-\sum X_i^2}=$$ $$\frac{\frac{\sum Y_i}{n} \cdot \sum X_i-\sum Y_i X_i}{-(\widehat{X} \cdot (n \widehat{X})-\sum (2\widehat{X} X_i)+\sum X_i^2)}=\frac{\widehat{X}\sum Y_i-\sum Y_i X_i}{-(\sum (\widehat{X})^2-\sum (2\widehat{X} X_i)+\sum X_i^2)}=$$ $$\color{blue}{\frac{\sum_{i=1}^n(X_i-\widehat{X})Y_i}{\sum_{i=1}^n(X_i-\widehat{X})^2}}$$

$\endgroup$
1
$\begingroup$

Remark: The common notation for sample average of $X$ is $\bar{X}$.

Notice that since $\sum_{i=1}^n (\bar{X}-X_i)=0$, we have $\sum_{i=1}^n \bar{X}(\bar{X}-X_i)=0$

\begin{align}\sum_{i=1}^n (\bar{X}X_i-X_i^2)&=\sum_{i=1}^n X_i(\bar{X}-X_i) \\ &=\sum_{i=1}^n X_i(\bar{X}-X_i) -\sum_{i=1}^n \bar{X}(\bar{X}-X_i) \\ &=\sum_{i=1}^n (X_i-\bar{X})(\bar{X}-X_i)\\ &=- \sum_{i=1}^n (X_i-\bar{X})^2\end{align}

Similar simplification can be performed for the numerator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.