If we have the following weighted least-squares regression, with

$\hat{\beta} = (X'WX)^{-1}X'WY$

How can we express the squared errors, MSE and the fitted values in matrix form?

These are the OLS equivalent:

$Squared Error_i = (\hat{y}_i - y_i )^2$

$Mean Squared Error(MSE) = \Sigma(\hat{y}_i - y_i)^2/(N-p)$

$Fitted Value_i = \hat{y}_i= \hat{\beta} * x_i$

Thank you!


One approach is to consider transformed variables. If you define e.g. $$ X^*=W^{1/2}X\\ Y^*=W^{1/2}Y $$ and apply this transformation, you can write your estimator as $$ \hat{\beta}=(X'WX)^{-1}X'WY=({X^*}'X^*)^{-1}{X^*}'Y^* $$ which is regular OLS, but it is applied to a transformed regression. Can you see what such a transformation means in terms of the scalar quantities you are looking for?

  • $\begingroup$ Not really actually, not in terms of $\hat {\beta}$ $\endgroup$ – Mayou Nov 16 '14 at 21:39
  • $\begingroup$ In other words, $\hat{Y^*} = X(X'WX)^{-1}X'WY$ or would it be $\hat{Y^*} = W^{1/2}X(X'WX)^{-1}X'WY$ ??? $\endgroup$ – Mayou Nov 17 '14 at 13:35
  • $\begingroup$ The $\hat\beta$ is still exactly the same, so $\hat Y=X\hat\beta=X(X'WX)^{-1}X'WY$. Weighted least squares like you're doing is equivalent to OLS of $Y^*=X^*\beta+e$, where $V(e)=\sigma^2 I$. Initially, you have $Y=X\beta+u$, where $V(u)=W^{-1}$. If you premultiply by $W^{1/2}$, you get $W^{1/2}Y=W^{1/2}X\beta+W^{1/2}u$ which we for simplicity can call $Y^*=X^*\beta+e$. So we transform the regression to get rid of the heteroskedasticity, and then what remains is a nice model in the sense that we can just apply OLS. $\endgroup$ – hejseb Nov 17 '14 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.