# How to take the derivative of quadratic term that involves vectors, transposes, and matrices, with respect to a scalar

What are the proper steps to take the derivative with respect to $\alpha$ of

$$\frac 1 2 (x - \alpha g)^T Q (x - \alpha g)$$

to get following?

$$-(x-\alpha g)^T Q g$$

(where $\alpha$ is a scalar, $Q$ is a symmetric positive definite matrix, and $x$ and $g$ are vectors of proper size.)

Background and further explanation:

I saw this differentiation as part of a differentiation of a larger expression in Nocedal and Wright's book Numerical Optimization, just above (3.25). The authors skip explaining differentiation steps and obtain the minimizer $\alpha$, as that is their purpose.

When I try to follow I see that the differentiation of this term must yield as above. Now my problem is, if I were to differentiate the term above I would not know that there would be a transpose there, or $Q$ would need to be in the middle. E.g. if everything were scalars the term would be $\frac 1 2 Q (x - \alpha g)^2$, and using the chain rule, I would obtain the derivative as $-Q(x-\alpha g)g$. Now that the vectors and transposes and matrices are involved, how should one apply the chain rule here properly?

• @anapprentice No problem :) Yeah $Q$ being symmetric means $Q=Q^T$, and since $x^T Q y$ is supposed to be a number say $r=x^T Q y$, then $r^T=r$, and so $r^T=y^TQ^Tx=y^TQx=x^TQy=r$ Nov 21 '17 at 13:52