Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Suppose $A$ is a $m\times n$ matrix, $x$ is a vector of unkowns and $b$ is a vector of know entries.

Why don't $$Ax=b$$ and $$A^TAx = A^Tb$$ have the same solution ($x$)?

It seems to me that I could get from the first equation to the second equation by simply multiplying both sides by $A^T$, so in my conception this wouldn't change the solutions.

Am I missing something?

share|improve this question
7  
In ordinary algebra do you expect $0x=1$ and $0\cdot0x=0\cdot1$ to have the same solutions? No, because $0$ has no multiplicative inverse. A similar situation obtains with matrix products: if $A$ is not invertible, that multiplication can introduce new solutions. –  Brian M. Scott Feb 18 '13 at 14:47

3 Answers 3

up vote 1 down vote accepted

It seems to me that I could get from the first equation to the second equation by simply multiplying both sides by $A^T$, so in my conception this wouldn't change the solutions.

It's not what you're missing, but what you're adding. You can indeed get from the first equation to the second by multiplying both sides by $A^T$. This means every solution to the first equation is also a solution to the second equation.

But you are trying to say more: you are trying to say every solution to the second equation is also a solution to the first equation. But that doesn't follow from your observation.

To make that conclusion, you need to find some way to get from the second equation to the first one, or some other argument that would imply a solution to the second is a solution to the first -- and such a thing may not exist, meaning that the second equation really does have solutions that are not solutions to the first equation.

share|improve this answer
    
Great answer, thank you. –  JLagana Feb 19 '13 at 22:35

Every solution to $Ax = b$ is a solution to $A^T A x = A^T b$, but not the other way around. A solution to $A^T A x = A^T b$ need not be a solution to $Ax = b$, unless $A^T$ is invertible (which is often not the case in applications).

share|improve this answer

If $A$ is invertible, they will indeed have the same solution because you could go from the second equation to the first by multiplying by $(A^{T})^{-1}$. In general, however, the first may have infinitely many solutions, whereas the second will have a unique solution if you have only linearly independent columns of $A$. See Moore-Penrose pseudoinverse for more.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.