Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose a symmetric matrix $L\in\mathbb{R}^{n\times n}$ is given, and a rectangular matrix $A\in\mathbb{R}^{n\times m}$, $m<n$. A solution to the system $$LAx=b, \tag 1$$ is sought, for known $b\in\mathbb{R}^n$ and unknown $x\in\mathbb{R}^m$. Since $LA\in\mathbb{R}^{n\times m}$, a solution $x$ should be calculated in a least squares sense. Now, suppose that one premultiplies the above system by $A^T$, ie, $$A^TLAx=A^Tb \tag 2$$

Under which conditions on $A$ (or $A^T$) can the solution to (2) be obtained by solving (1) (by eliminating $A^T$ from the second)? In other words, is it safe to replace solving (1) by solving (2), or vice versa?

I suppose that in case $A^T$ is non-invertible the solution to the first and the second system differ. What is the condition in a general case of rectangular $A$? What about square symmetric $A=A^T$?


The question reduces to: could one replace solving (1) by solving (2), or vice versa? Why couldn't one replace solving (2) by solving (1)? If a solution to (1) exists, then it is also a solution to (2). Could it be that the solution to (1) does not exits (ie, exists only in LS sense), but the exact solution to (2) exists?

share|cite|improve this question
The second form may not have exact solutions if $\mathbf{A}^T\mathbf{L}\mathbf{A}$ is not full rank. – chaohuang Sep 17 '12 at 18:15
@chaohuang Could you make a more elaborate answer (considering my questions)? Does this mean that the solution to the first form and the one to the second are actually the same, and the minimization in the least square sense is inherited by the second form? – user506901 Sep 17 '12 at 18:19
The least squares solution of the first form is the same as the one to its normal equation, which is $A^TL^TLAx=A^TL^Tb$, not $A^TLAx=A^Tb$ – chaohuang Sep 17 '12 at 19:55
@chaohuang Why is not a solution to $LAx=b$ (existent in a least squares sense) also a solution to $A^TLAX=A^Tb$? – user506901 Sep 18 '12 at 13:26
@user506901 Let $LA = C$. $C$ is a non-square matrix, because $A$ is non-square. Then your problem is solving $Cx = b$. To do this in the LS sense, you pre-multiply by $C^T$ and solve the resulting system: $$ C^TCx = C^Tb \Longrightarrow (LA)^TLAx = (LA)^Tb$$ Of course, $(LA)^TLA = A^TL^TLA$, which is not necessarily equal to $A^TLA$. – Emily Sep 19 '12 at 14:21

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.