0
$\begingroup$

"Consider the standard model:

$$ Y = X \beta + \epsilon $$

where $X$ is the $n \times p$ matrix of rank $p$, $\beta$ is vector of $p$ unknown parameters and $\epsilon$ is a random vector whose components are independent normal random variables, each with mean $0$ and variance $\sigma^2$. For the least squares estimator $\hat{\beta} = (X^T X)^{-1} X^T Y$ of $\beta$, denote the vector of residuals by $r = Y - X \hat{\beta}$. Show that the residual sum of squares fulfills:

$$ r^Tr = Y^TY - Y^TX \hat{\beta}. $$""

How do I go about doing this. I managed to prove that $X^TR = 0$ but I'm not really sure what to do from there.

$\endgroup$
0
$\begingroup$

We have \begin{align*} r^\top r &= Y^\top Y - Y^\top X\hat \beta + \hat\beta^\top X^\top r \end{align*} Now \begin{align*} X^\top r &= X^\top Y - X^\top X \hat \beta\\ &= X^\top Y - X^\top X \hat \beta\\ &= X^\top Y - X^\top X (X^\top X)^{-1} X^\top Y\\ &= 0 \end{align*} Hence from the above \begin{align*} r^\top r &= Y^\top Y - Y^\top X\hat \beta + \hat\beta^\top X^\top r = Y^\top Y - Y^\top X\hat \beta \end{align*}

$\endgroup$
  • $\begingroup$ Why do you have the $+\hat{\beta}^TX^Tr$ in the first equation? Where does that come from? $\endgroup$ – Kaish Oct 27 '12 at 22:05
  • $\begingroup$ We have $r = Y - X\hat \beta$, so $r^\top = Y^\top -\hat\beta^\top X^\top$ and hence $r^\top r = Y^\top (Y - X\hat\beta) - \hat\beta^\top X^\top r$. $\endgroup$ – martini Oct 28 '12 at 8:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.