4
$\begingroup$

I have something like this $$X^T A^T A X - X^T A^T B - B^T A X$$ or $$X^T A^T A X -2 X^T A^T B$$ since X and B are really vectors ($X^TA^TB=B^TAX$).

I’d like to find the X that minimizes the expression. To do that I want to have something like this $$[X-(A^TA)^{-1}A^TB]^T(A^TA)[X-(A^TA)^{-1}A^TB]-B^TA(A^TA)^{-1}A^TB$$ so that the X that minimizes the expression is $(A^TA)^{-1}A^TB$.

My question is: how does all this relate to the technique of completing the squares? How can I show the squares?

To further explain my question, it’s easy to say, if you have $a^2 + 2ab$ just add and subtract $b^2$. But how can I abstract that to matrices?

PS: I assume $A^TA$ is non singular, positive definite.

$\endgroup$
4
  • $\begingroup$ (i) What is "C"? Is the C in line 5 supposed to be B? (ii) Since matrix multiplication is not commutative, $(A+B)^2 = A^2 + AB + BA + B^2$. If you have $A^2+2AB$, simply adding and subtracting $B^2$ will not give you $(A+B)^2-B^2$. $\endgroup$
    – Arturo Magidin
    Jun 12, 2011 at 4:33
  • $\begingroup$ As written, the first equation does not make sense. The first term is a scalar, but the other terms are vectors, presuming that $A$ and $B$ are matrices. Does the missing $C$ fix that? In truth, none of equations make sense for the same reason. $\endgroup$
    – rcollyer
    Jun 12, 2011 at 4:39
  • $\begingroup$ Sorry I’ll fix the question. $\endgroup$
    – gurghet
    Jun 12, 2011 at 4:42
  • $\begingroup$ In my question there is not a problem to be solved, more like a solution to be explained. Going to the first equation to the second equation (the long one) causes me much trouble. $\endgroup$
    – gurghet
    Jun 12, 2011 at 5:07

1 Answer 1

Reset to default
5
$\begingroup$

Write $$X^T A^T A X - X^T A^T B - B^T A X = (A X - B)^T (A X - B) - B^T B.$$

So you want to minimize $(A X - B)^T (A X - B)$.

Note that $A (A^T A)^{-1} A^T$ is the orthogonal projection on ${\rm Ran}(A)$. Thus $B = A u + v$ where $u = (A^T A)^{-1} A^T B$, and $v \perp A x$ for all $x$.

Now $A X - B = A(X - u) - v$, so $(A X - B)^T (A X - B) \ge v^T v$. Thus the minimum value of $(A X - B)^T (A X - B)$ is $v^T v$, attained for $X = u$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.