1
$\begingroup$

While I am able to see the differentiation of a matrix expression in the matrix cookbook of this form,

$$ \frac{\partial \mathbf{b}^T \mathbf{X}^T\mathbf{X}\mathbf{c}}{\partial \mathbf{X}} = \mathbf{X} (\mathbf{b} \mathbf{c}^T + \mathbf{c} \mathbf{b}^T)$$

I am unable to figure out the derivative of the numerator's transpose from the cookbook i.e.

$$ \frac{\partial \mathbf{c}^T \mathbf{X}\mathbf{X}^T \mathbf{b}}{\partial \mathbf{X}} = \ ?$$

$\endgroup$
  • $\begingroup$ Note: $(b^TX^TXc)^T = c^T(b^TX^TX)^T = c^T(X^TX)^Tb = c^T(X^TX)b$ which is not the same as $bXX^Tc^T$ $\endgroup$ – EDZ Feb 22 '18 at 19:10
  • 1
    $\begingroup$ The above can be seen because if $X$ is an $n x m$ matrix, $X^TX$ is an $m x m$ matrix whose transpose is $m x m$ and $XX^T$ is an $n x n$ matrix whose transpose is also an $n x n$ matrix. For an actual example, use: $ X = \begin{bmatrix} 3 & 1 \\ -1 & 5\\ 4 & 2 \end{bmatrix}$ $\endgroup$ – EDZ Feb 22 '18 at 19:23
  • $\begingroup$ If you consider column vectors $b$ and $c$, then $bXX^Tc^T$ is likely not well defined $\endgroup$ – user251257 Feb 22 '18 at 19:49
1
$\begingroup$

For $f(X) = b^T X X^T c$ we have $$Df(X)[H] = b^T H X^T c + b^T X H^T c = tr(X^T cb^T H) + tr(X^Tbc^TH) .$$ So we have $$ \frac{\partial f(X)}{\partial X} = (bc^T + cb^T) X .$$

$\endgroup$
  • $\begingroup$ I understand the concept of directional derivative and the cyclic permutation you did to proof but can you please share some resource that can help me understand the concept better and help me solve more complicated expressions like above. $\endgroup$ – learner Feb 22 '18 at 23:00
  • 1
    $\begingroup$ @learner I only used the product rule. Basically you can do most stuff with product rule, chain rule and inverse function theorem. $\endgroup$ – user251257 Feb 22 '18 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.