3
$\begingroup$

What is the derivative (gradient) of $$\operatorname{vec}(W)^\top \left[\sum_i (ue_i^\top)\otimes(e_ie_i^\top) + \sum_j(e_je_j^\top)\otimes(ue_j^\top) - 2I\otimes I\right] \operatorname{vec}(FF^\top),\tag{3} $$ where $e_i$ denotes the $i$-th vector in the canonical basis and $u=\sum_ie_i=(1,1,\ldots,1)^\top$ where "$\otimes$" denotes the Kronecker product (tensor product) with respect to the rectangular matrix $F$? and $W$ is a constant matrix. The above term can also be simplified as $\sum_{ijl}(F_{il}-F_{jl})^2W_{ij}$ which is a much simpler expression. What would the derivative be? All the above entries are real while $F$ is a rectangular matrix. Also please refer to : Represent in a matrix form: $\sum_{ijl}(F_{il}-F_{jl})^2W_{ij}$ for the details on simplification/algebra.

$\endgroup$
2
  • 3
    $\begingroup$ The whole left side of the expression is independent of $F$, so all you need to do is calculate the vector derivative of $\mathrm{vec}(FF^T)$. $\endgroup$ May 28, 2013 at 7:17
  • 3
    $\begingroup$ Considering the expression on the left to be $C$, the derivative of $CFF^T$ would be $(C+C^T)F$, I guess? Now with $C\operatorname{vec}(FF^T)$, am confused.. $\endgroup$
    – user75402
    May 28, 2013 at 16:15

1 Answer 1

2
$\begingroup$

To find the derivative, it is much easier to work with your original expression $S=\sum_{ijl}(F_{il}-F_{jl})^2W_{ij}$. Note that, for fixed indices $I$ and $J$, \begin{align*} S = &\text{ terms that do not involve } F_{IJ}\\ &+\ (F_{IJ} - F_{IJ})^2 W_{II} + \sum_{j\neq I} (F_{IJ} - F_{jJ})^2 W_{Ij} + \sum_{i\neq I} (F_{iJ} - F_{IJ})^2 W_{iI}\\ = &\text{ terms that do not involve } F_{IJ}\\ &+\ \sum_{k\neq I} (F_{IJ} - F_{kJ})^2 W_{Ik} + \sum_{k\neq I} (F_{kJ} - F_{IJ})^2 W_{kI}. \end{align*} Let $K = W-W^T$. Then \begin{align*} \frac{\partial S}{\partial F_{IJ}} =&2\sum_{k\neq I} (F_{IJ} - F_{kJ}) W_{Ik} + 2\sum_{k\neq I} (F_{kJ} - F_{IJ}) W_{kI}\\ =&2\sum_{k\neq I} (F_{IJ} - F_{kJ}) (W_{Ik}-W_{kI})\\ =&2\sum_{k\neq I} (F_{IJ} - F_{kJ}) K_{Ik}\\ =&2\sum_k (F_{IJ} - F_{kJ}) K_{Ik}\\ =&2F_{IJ}\sum_k K_{Ik} - 2\sum_k K_{Ik}F_{kJ}. \end{align*} Therefore $$ \left(\frac{\partial S}{\partial F_{ij}}\right) =2F\circ(KE) - 2KF $$ where $E$ is the matrix with the same size as $F$ and all entries equal to $1$.

$\endgroup$
4
  • $\begingroup$ Wonderful answer! $\endgroup$
    – user75402
    May 29, 2013 at 13:51
  • $\begingroup$ also that's the Hadamard Product between $F$ and $KE$. Right? $\endgroup$
    – user75402
    Jun 13, 2013 at 20:39
  • $\begingroup$ @user75402 Yes, it's the Hadamard product. $\endgroup$
    – user1551
    Jun 14, 2013 at 6:04
  • $\begingroup$ There is a sign error in the second term of the derivative. The derivative of $(F_{kJ}-F_{IJ})^2$ with respect to $F_{IJ}$ is $-2\,(F_{kJ}-F_{IJ})$. However, the form of the final result is correct, if you redefine $K=W+W^T$. $\endgroup$
    – greg
    May 2, 2015 at 23:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .