2
$\begingroup$

Given the matrix

$$ A = \begin{bmatrix}1 & 1 \\ 2 & -1 \\ -2 & 4\end{bmatrix} $$

I found the corresponding orthogonal matrix using the Gram-Schmidt process

$$ Q = \frac{1}{3} \begin{bmatrix}1 & 2 \\ 2 & 1 \\ -2 & 2\end{bmatrix} $$

If I try to solve $Ax = (1, 2, 7)$ by least squares with $A$, I get

$$ \hat{x}_1 = (A^TA)^{-1}A^Tb = \frac{1}{9} \begin{bmatrix}2 & 1 \\ 1 & 1\end{bmatrix} \begin{bmatrix}1 & 2 & -2 \\ 1 & -1 & 4\end{bmatrix} \begin{bmatrix} 1 \\ 2 \\ 7\end{bmatrix} = \begin{bmatrix}1 \\ 2\end{bmatrix} $$

but if I try with $Q$, letting $Q^TQ = I$, I get

$$ \hat{x}_2 = Q^Tb = \frac{1}{3} \begin{bmatrix} 1 & 2 & -2 \\ 2 & 1 & 2\end{bmatrix} \begin{bmatrix} 1 \\ 2 \\ 7\end{bmatrix} = \begin{bmatrix}-3 \\ 6\end{bmatrix} $$

I was expecting the vectors $\hat{x}_1$ and $\hat{x}_2$ to be the same or at least be in the same subspace; they're not on the same line and they're not orthogonal either. Why are the solutions are so different?

My apologies if the question is a bit vague but there's something I clearly don't understand.

$\endgroup$
0
1
$\begingroup$

The point with coordinates $$A\hat{x}_1 = Q\hat{x}_2 = \begin{bmatrix}3 \\ 0 \\ 6\end{bmatrix}$$ is the projection of $(1,2,7)$ on the plane $ColSp(A) = ColSp(Q)$.

$\endgroup$
2
  • $\begingroup$ That clarified a lot! If I factor $A = QR$, I get $R = \begin{bmatrix}3 & 0 \\ 0 & 3\end{bmatrix}$. Can that be used somehow to map between $\hat{x}_1$ and $\hat{x}_2$? $\endgroup$ Jun 16 '16 at 1:19
  • 1
    $\begingroup$ $Q\hat{x}_2 = A\hat{x}_1 = QR\hat{x}_1$ and since the columns of $Q$ are linearly independent, that implies $R\hat{x}_1 = \hat{x}_2$. (But the correct $R$ has a -3 as (1,2) entry.) $\endgroup$ Jun 16 '16 at 1:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.