# Prove $A^T(A\hat{\mathbf{x}}-\mathbf{b}) = \mathbf{0}$.

There's a question in a textbook I've just looked at. Let $$A$$ be an $$m\times n$$ matrix. Prove that for all $$\mathbf{b}\in \mathbb{R}^m$$ there exists $$\hat{\mathbf{x}}\in \mathbb{R}^n$$ for which $$A^T(A\hat{\mathbf{x}}-\mathbf{b}) = \mathbf{0}$$.
Hint: $$\text{col}(A)+\text{col}(A)^{\perp} = \mathbb{R}^m$$.
Just wondering how to go about it. Thanks!

• Fun trivia: This calculation lies at the core of least-squares linear regression. The $\hat{\mathbf x}$ you find here is the least-square-error "solution" to the potentially inconsistent equation $A\mathbf x=\mathbf b$. – Arthur Aug 14 at 7:41
• Choose $\hat x$ so that $A \hat x$ is as close as possible to $b$. Visually, the residual $r = b - A \hat x$ most be orthogonal to the column space of $A$. In particular, $r$ is orthogonal to each column of $A$. (This is the key intuition behind the normal equations for least squares problems.) – littleO Aug 14 at 7:44
• So we are just deriving $A^TA\hat{\mathbf{x}} = A^T\mathbf{b}$ thanks. Just wasn’t sure how to use the hint or even if we have to! – squenshl Aug 14 at 19:01