Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose I have the linear regression model $ \hat{y_i} = a + b x_i $ for $a,b$ obtained via OLS. How does one regress $y$ back on the residuals $\hat{e}_i = y_i - \hat{y}_i$? If we write $ \hat{\hat{y_i}} = c + d \hat{e}_i$ and attempt to use the regression coefficient formulas, I'm unsure how to write $c$ and $d$ in terms of $a,b$.

share|cite|improve this question
Hint: When you think of $y$, $x$ and $\hat{e}$ as vectors, the normal equations assert that $\hat{e}$ is orthogonal to $x$. When (after centering $x$ and $y$) you write $y = 1 \cdot \hat{e} + b x$, the residuals--thinking of $\hat{e}$ as the variables this time--are $b x$. If you could somehow show $b x$ is orthogonal to $\hat{e}$, this would demonstrate that you have indeed regressed $y$ against $\hat{e}$. – whuber Dec 27 '12 at 22:59
I'm still having some trouble. I understand the orthogonality property, but the equations I am finding doesn't seem correct. – Peter Dec 28 '12 at 2:53

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.