How to prove differentiability of $g(x)=x^TAx$?

What I've started with is the definition of differentiability:

Let $G \subset \mathbb{R}^n$ be open. $g:G \rightarrow \mathbb{R}^n$ is differentiable at $x \in G$ if exists a linear transformation $L: \mathbb{R}^n \rightarrow \mathbb{R}^m$ s.t.

$$\lim_{h \rightarrow 0} \frac{g(x+h)-g(x)-Lh}{||h||}=0$$

Now since $x^TAx$ is not a linear form, then I don't understand how can do anything with the above definition.

  • 1
    $\begingroup$ Related: math.stackexchange.com/questions/312077/… $\endgroup$ – Arthur Nov 17 '16 at 9:46
  • $\begingroup$ rewrite $x^T A x$ as $\sum_{i, j} a_{ij} x_i x_j$ $\endgroup$ – Denis Korzhenkov Nov 17 '16 at 9:47
  • $\begingroup$ @Arthur Yes, but I'm confused about what to do with $$\lim_{h \rightarrow 0} \frac{2x^TA \cdot h + h^TAh - Lh}{||h||}$$ that results from that first answer. Is $\frac{h}{||h||}$ perhaps something special? $\endgroup$ – mavavilj Nov 17 '16 at 9:52
  • $\begingroup$ Or maybe I can apply Squeeze theorem? $\endgroup$ – mavavilj Nov 17 '16 at 10:01

Here, $g(x+h) = (x+h)^T A (x+h) = x^TAx + x^TAh + h^TAx + h^TAh$.

So that $$\frac{g(x+h)-g(x)}{||h||} = \frac{x^TAh + h^TAx + h^TAh}{||h||}$$.

Let $L(h) = x^TAh + h^TAx$. You can check that this is linear (in $h$).

Then, $$\frac{g(x+h)-g(x)-L(h)}{||h||} = \frac{h^TAh}{||h||}$$

Finally, note that $\lim_{h \to 0}\frac{h^TAh}{||h||} = 0$. Hence, the linear map $L(h) = x^TAh + h^TAx$ is the derivative of the map.

The way of picking $L$ was simple: I just took out all the linear terms of $h$ from the numerator using $L$. What remained were higher order terms, and these dominate $||h||$ as the quotient goes to zero. Hence, $L$ is very logically picked.

There are many ways of doing this problem. One is via what Denis suggested above. Another is via chain rule. But this is the elementary method.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.