Sum of residuals proof Show that: $\sum x_i e_i=0$ and also show that $\sum\hat{y}_i e_i=0$. Now I do believe that being able to solve the first sum will make the solution to the second sum more clear. So far I have proved that $\sum e_i=0$. Any hints would be helpful!
Edit: $e_i$ are the residuals.
Thanks!
 A: Here's one way of viewing it.  We want to write
$$
\begin{bmatrix} y_1 \\  \vdots \\  y_n \end{bmatrix} = \hat\alpha\begin{bmatrix} 1 \\  \vdots \\ 1 \end{bmatrix} + \hat\beta \begin{bmatrix} x_1 \\  \vdots \\ x_n \end{bmatrix} + \begin{bmatrix} \hat\varepsilon_1 \\  \vdots \\  \varepsilon_n \end{bmatrix}
$$
and choose the values of $\hat\alpha$ and $\hat\beta$ that minimze $\hat\varepsilon_1^2+\cdots+\hat\varepsilon_n^2$.  The sum of the first two terms on the right is $[ \hat y_1,\ldots,\hat y_n]^T$.
That means the point $\begin{bmatrix} \hat y_1 \\  \vdots \\  y_n \end{bmatrix} = \hat\alpha\begin{bmatrix} 1 \\  \vdots \\ 1 \end{bmatrix} + \hat\beta \begin{bmatrix} x_1 \\  \vdots \\ x_n \end{bmatrix}$ is closer to $\begin{bmatrix} y_1 \\  \vdots \\  y_n \end{bmatrix}$ in ordinary Euclidean distance than is any other point in the plane spanned by $\begin{bmatrix} 1 \\  \vdots \\ 1 \end{bmatrix}$ and $\begin{bmatrix} x_1 \\  \vdots \\ x_n \end{bmatrix}$.  The point in a plane that is closet to $\mathbf y$ is the point you get by dropping a perpendicular from $\mathbf y$ to the plane.  That means $\hat\varepsilon=\hat{\mathbf y} - \mathbf y$ is perpendicular to the two columns that span the plane, and thus perpendicular to every linear combination of them, such as $\hat{\mathbf y}$.  "Perpendicular" means the dot-product is zero. Q.E.D.
I can also write a more algebraic argument.  Maybe I'll attend to that tomorrow.
Continued on Friday: The vector of fitted values $\hat{\mathbf y}=\begin{bmatrix} \hat y_1 \\  \vdots \\ \hat y_n \end{bmatrix}$ is the orthogonal projection of the vector $\mathbf y = \begin{bmatrix} y_1 \\  \vdots \\ y_n \end{bmatrix}$ onto the column space of the design matrix $X=\begin{bmatrix} 1 & x_1 \\  \vdots & \vdots \\ 1 & x_n \end{bmatrix}$.
The orthogonal projection is a linear transformation whose matrix is the "hat matrix" is $H=X(X^T X)^{-1}X^T$, an $n\times n$ matrix of rank $2$.  Observe that if $\mathbf w$ is orthogonal to that column space then $X{\mathbf w}=0$ so $H{\mathbf w}=0$, and if $\mathbf w$ is in the column space, then ${\mathbf w}=Xu$ for some $u\in\mathbb R^2$, and so $H\mathbf w=\mathbf w$.
It follows that $\hat\varepsilon = \mathbf y - \hat{\mathbf y}=(I-H)\mathbf y$ is orthogonal to the column space.  Since $\hat{\mathbf y}$ is in the column space, $\hat\varepsilon$ is orthogonal to $\hat{\mathbf y}$.
