3
$\begingroup$

If we have the following weighted least-squares regression, with

$\hat{\beta} = (X'WX)^{-1}X'WY$

How can we express the squared errors, MSE and the fitted values in matrix form?

These are the OLS equivalent:

$Squared Error_i = (\hat{y}_i - y_i )^2$

$Mean Squared Error(MSE) = \Sigma(\hat{y}_i - y_i)^2/(N-p)$

$Fitted Value_i = \hat{y}_i= \hat{\beta} * x_i$

Thank you!

$\endgroup$

1 Answer 1

1
$\begingroup$

One approach is to consider transformed variables. If you define e.g. $$ X^*=W^{1/2}X\\ Y^*=W^{1/2}Y $$ and apply this transformation, you can write your estimator as $$ \hat{\beta}=(X'WX)^{-1}X'WY=({X^*}'X^*)^{-1}{X^*}'Y^* $$ which is regular OLS, but it is applied to a transformed regression. Can you see what such a transformation means in terms of the scalar quantities you are looking for?

$\endgroup$
3
  • $\begingroup$ Not really actually, not in terms of $\hat {\beta}$ $\endgroup$
    – Mayou
    Commented Nov 16, 2014 at 21:39
  • $\begingroup$ In other words, $\hat{Y^*} = X(X'WX)^{-1}X'WY$ or would it be $\hat{Y^*} = W^{1/2}X(X'WX)^{-1}X'WY$ ??? $\endgroup$
    – Mayou
    Commented Nov 17, 2014 at 13:35
  • $\begingroup$ The $\hat\beta$ is still exactly the same, so $\hat Y=X\hat\beta=X(X'WX)^{-1}X'WY$. Weighted least squares like you're doing is equivalent to OLS of $Y^*=X^*\beta+e$, where $V(e)=\sigma^2 I$. Initially, you have $Y=X\beta+u$, where $V(u)=W^{-1}$. If you premultiply by $W^{1/2}$, you get $W^{1/2}Y=W^{1/2}X\beta+W^{1/2}u$ which we for simplicity can call $Y^*=X^*\beta+e$. So we transform the regression to get rid of the heteroskedasticity, and then what remains is a nice model in the sense that we can just apply OLS. $\endgroup$
    – hejseb
    Commented Nov 17, 2014 at 15:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .