2
$\begingroup$

Self studying linear algebra here.

Source: James E. Gentle, Matrix Algebra Theory..., 2007, p. 151

Given: $x^TAx$ is a quadratic form. $y$ is a vector. $t$ is a scalar.

Evaluate $\frac{\partial x^TAx}{\partial x}$ using the limit.

Here is what I came up with:

$\lim_{t \to 0}\frac{(x+ty)^TA(x+ty)-x^TAx}{t}$

Being pedantic, including this next intermediary step for illustrative purposes (just trying to give more surface area so you can point out any misunderstandings of mine wrt matrix operations, etc.):

$=\lim_{t \to 0}\frac{(x+ty)^T(Ax+Aty)-x^TAx}{t} $ (expand the 2nd and 3rd factors of the numerator first, could have first expanded 1st and 2nd instead)

$=\lim_{t \to 0}\frac{x^TAx+x^TAty+ty^TAx+ty^TAty-x^TAx}{t} $

$=\lim_{t \to 0}\frac{tx^TAy+ty^TAx+t^2y^TAy}{t}$ ($x^TAx-x^TAx$ cancel, $t$'s move to the front of each term.)

$=\lim_{t \to 0}\frac{x^TAy+y^TAx+2ty^TAy}{1} $ ($t$'s cancel)

$=x^TAy+y^TAx $

Update (continuing):

$=y^TA^Tx+y^TAx$ (thanks Christopher A. Wong, I somehow missed this rule. I should have done a small experiment to work it out.)

$=y^T(A^T+A)x$

The first expansion is supposed to be:

$=\lim_{t \to 0}\frac{x^TAx+ty^TAx+ty^TA^Tx+t^2y^TAy-x^TAx}{t} $

The answer is supposed to be:

$y^T(A+A^T)x$

Question: How to get $y^T(A+A^T)x$ step by step.

$\endgroup$
  • 7
    $\begingroup$ You're almost there. Use $x^T Ay = y^T A^T x$. This follows from the fact that the transpose of a scalar is itself. $\endgroup$ – Christopher A. Wong Feb 19 '14 at 0:06
  • 2
    $\begingroup$ I don;t see what L'Hôpital's rule has to do with anything. That step is just a simplification by $t$. $\endgroup$ – jathd Feb 19 '14 at 0:08
  • $\begingroup$ Thanks Christopher A. Wong. $\endgroup$ – Joe Feb 19 '14 at 0:25
  • $\begingroup$ jathd, from my rusty memory of taking limits, you can't have the denominator going to zero (without the numerator going to zero at the same rate) as that would be an indefinite form. $\endgroup$ – Joe Feb 19 '14 at 0:26
  • $\begingroup$ jathd, you are correct, I really was rusty, there was a t in each term that can cancel with the denominator. My bad. $\endgroup$ – Joe Feb 26 '14 at 4:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.