Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

enter image description here

Assume that $A^\mathrm{T}A$ is invertible and that $A$ is $3\times 3$, $B$ is $3\times 1$, $\alpha$ is $3\times 1$. How does the expression above reduce?

just to $A^\mathrm{T}B$ ?

share|cite|improve this question
The information you've given is contradictory: $A$ cannot be both invertible and non-square. – Hurkyl Oct 12 '12 at 22:41
A $\,3\times 1\,$ matrix (or vector, as you wish to look at it) – DonAntonio Oct 12 '12 at 22:42
@Hurkyl We can, however, assume that the $3\times 3$ matrix $A^TA$ is invertible (which is what we're inverting anyways). Unless I've forgotten some important theorem. – Arthur Oct 12 '12 at 22:47
If $A$ is rectangular, it doesn't reduce further. – Ganesh Oct 12 '12 at 22:50
hmm you guys are right, that makes no sense. If we assume that ATA is invertible, then what does this reduce to? – mugetsu Oct 12 '12 at 23:13
up vote 2 down vote accepted

If $A$ is square and $A^tA$ is invertible then of course $A$ is invertible, and your expression simplifies to $A^{-1}B$.

share|cite|improve this answer
could you explain the steps for me? thanks – mugetsu Oct 14 '12 at 3:25
You know that $(XY)^{-1}=Y^{-1}X^{-1}$? Apply that to $(A^tA)^{-1}$. – Gerry Myerson Oct 14 '12 at 4:18

The expression itself does not reduce (though there are more complicated methods that are efficient at solving it). I believe your equation may be based on the desire to solve $$A\vec{\alpha}=B$$ given $A$ and $B$, with $A$ underdetermined, meaning it has fewer columns than rows, which is consistent with your problem statement of $5$ rows and $3$ columns for $A$. This equation has no exact solution, thus the next best thing is sought.

As there is no exact solution, consider that a solution that is the "closest" of all possible. With the notion of "close" as the Euclidean distance, it becomes what is called a Least Squares equation. To give you the very abbreviated version of Least Squares, left multiply both sides of the equation to get $$A^TA\vec{\alpha}=A^TB$$ If you are not yet familiar with Least Squares, for now know that the reason for this is to have something invertible to work with ($A^TA$ will have an inverse) and to keep things in terms of the range of $A$. Thus, to solve it, again left multiply both sides of the equation to get $$(A^TA)^{-1}A^TA\vec{\alpha}=(A^TA)^{-1}A^TB$$ which reduces to $$\vec{\alpha}=(A^TA)^{-1}A^TB$$

This equation then has a solution, and it can be shown to be the closest solution of $\vec{\alpha}$ to the original equation $A\vec{\alpha}=B$

Back again to your original question if it is reduceable; the answer is no-- though there are ways of going about solving this without calculating every middle step as I have shown here, and that would take quite a bit more explaining than I have done thus far.

share|cite|improve this answer
thanks for the help. I think i misinterpreted the problem and now I'm going with A as a 3x3 matrix – mugetsu Oct 13 '12 at 0:08
The only way for the equation to reduce is if $A$ actually is invertible, then it would reduce to $\alpha=A^{-1}B$ – adam W Oct 13 '12 at 0:29

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.