# Why is solution different for least squares with original matrix and orthogonal matrix?

Given the matrix

$$A = \begin{bmatrix}1 & 1 \\ 2 & -1 \\ -2 & 4\end{bmatrix}$$

I found the corresponding orthogonal matrix using the Gram-Schmidt process

$$Q = \frac{1}{3} \begin{bmatrix}1 & 2 \\ 2 & 1 \\ -2 & 2\end{bmatrix}$$

If I try to solve $Ax = (1, 2, 7)$ by least squares with $A$, I get

$$\hat{x}_1 = (A^TA)^{-1}A^Tb = \frac{1}{9} \begin{bmatrix}2 & 1 \\ 1 & 1\end{bmatrix} \begin{bmatrix}1 & 2 & -2 \\ 1 & -1 & 4\end{bmatrix} \begin{bmatrix} 1 \\ 2 \\ 7\end{bmatrix} = \begin{bmatrix}1 \\ 2\end{bmatrix}$$

but if I try with $Q$, letting $Q^TQ = I$, I get

$$\hat{x}_2 = Q^Tb = \frac{1}{3} \begin{bmatrix} 1 & 2 & -2 \\ 2 & 1 & 2\end{bmatrix} \begin{bmatrix} 1 \\ 2 \\ 7\end{bmatrix} = \begin{bmatrix}-3 \\ 6\end{bmatrix}$$

I was expecting the vectors $\hat{x}_1$ and $\hat{x}_2$ to be the same or at least be in the same subspace; they're not on the same line and they're not orthogonal either. Why are the solutions are so different?

My apologies if the question is a bit vague but there's something I clearly don't understand.

The point with coordinates $$A\hat{x}_1 = Q\hat{x}_2 = \begin{bmatrix}3 \\ 0 \\ 6\end{bmatrix}$$ is the projection of $(1,2,7)$ on the plane $ColSp(A) = ColSp(Q)$.
• That clarified a lot! If I factor $A = QR$, I get $R = \begin{bmatrix}3 & 0 \\ 0 & 3\end{bmatrix}$. Can that be used somehow to map between $\hat{x}_1$ and $\hat{x}_2$? Jun 16 '16 at 1:19
• $Q\hat{x}_2 = A\hat{x}_1 = QR\hat{x}_1$ and since the columns of $Q$ are linearly independent, that implies $R\hat{x}_1 = \hat{x}_2$. (But the correct $R$ has a -3 as (1,2) entry.) Jun 16 '16 at 1:26