1
$\begingroup$

I am attempting to find the least-squares solution x* of the system Ax = b, where $$ A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \\ \end{bmatrix} $$ and $$ B = \begin{bmatrix} 1 \\ 0 \\ 0 \\ \end{bmatrix} $$ I'm trying to solve for x* using the formula $$x^* = (A^TA)^{-1} A^Tb $$ I am getting stuck after I calculate $$(A^TA)= \begin{bmatrix} 66 & 78 & 90 \\ 78 & 93 & 108 \\ 90 & 108 & 126 \\ \end{bmatrix} $$ because I am getting an error when I attempt to calculate the inverse of this matrix in my calculator. What am I missing?

$\endgroup$
1
$\begingroup$

The determinant of $A$ is $0$, hence it is not invertible.

$\endgroup$
1
  • $\begingroup$ So then does that mean it is impossible to calculate a least-squares solution? $\endgroup$ Mar 26 '15 at 2:39
1
$\begingroup$

The matrix $A^TA$ is not full rank because the matrix $A$ is not full rank. This is why you cannot invert it. Re-express the system using only linearly independent rows. The matrix $A$ is of rank 2. Re-express your system with a $2\times 3$ matrix. With this representation, you can use the least square formula.

$\endgroup$
1
  • $\begingroup$ Agh, I get it now! I need to ignore two of the columns in A and then plug that 3x1 matrix into the formula. I've run some regressions in Stata before, but I only just realized why a regression that includes duplicate variables omits duplicate occurrences. $\endgroup$ Mar 26 '15 at 13:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.