1
$\begingroup$

Suppose $P$ is a real-valued function of the $p\times m$ (real) matrix $\mathbf{Q}$. After taking its differential, one arrives with the following:

$$ d(P(\mathbf{Q})) = \operatorname{trace}\left\{\mathbf{1}^\top_p\left[ d\mathbf{Q}\odot \mathbf{W} \right]\left(\mathbf{w}\odot \mathbf{G}\right)\right\} $$ where $\mathbf{1}_p$ is the $p\times 1$ vector of $1$'s with $\mathbf{W}$ is $p\times m$ while $\mathbf{w}$ and $\mathbf{G}$ are both $m \times 1$. $\mathbf{W}$, $\mathbf{w}$ and $\mathbf{G}$ are matrices involving $\mathbf{Q}$.

Question: What is $ \dfrac{dP}{d\mathbf{Q}} $ ?

Attempt: $ \dfrac{dP}{d\mathbf{Q}} = \mathbf{W} \left(\mathbf{w}\odot \mathbf{G}\right) \mathbf{1}_p $

But I think it's wrong. So my problem really is that Hadamard product of $d\mathbf{Q}$ and $\mathbf{W} $.

Some identities I have found online are these:

• $\dfrac{d(\mathbf{a}^\top\mathbf{X}\mathbf{b})} {d \mathbf{X}} = \mathbf{a} \mathbf{b}^\top$

• $\operatorname{trace} (\mathbf{A}\odot \mathbf{B})\mathbf{C} = \operatorname{trace} \mathbf{A} (\mathbf{B}^\top \odot \mathbf{C})$

UPDATE: To make it simpler, a general problem would be

$$ \frac{\mathbf{a}^\top\left[d\mathbf{Q}\odot f(\mathbf{Q}) \right]g(\mathbf{Q})}{d\mathbf{Q}} $$ where $\mathbf{a}\in \mathbb{R}^{p}$, $f:\mathbb{R}^{p\times m}\rightarrow \mathbb{R}^{p\times m}$ and $g:\mathbb{R}^{p\times m}\rightarrow \mathbb{R}^{m}$.

The available identity I have encountered similar to this is

$$ \frac{\operatorname{trace}(\mathbf{A}d\mathbf{X})}{d\mathbf{X}} = \mathbf{A} $$

from page 2 of this link.

$\endgroup$
2
  • 1
    $\begingroup$ Note that $1_p^T$ is a row vector and everything to the right of it is a matrix (lump it together and call it $M$). The quantity $r^T=1_p^TM$ is also a row vector. How do you define the trace of a row vector? $\endgroup$
    – greg
    Commented Mar 22, 2019 at 14:41
  • $\begingroup$ Thank you @greg for point out. I correted it already together with an attempt. $\endgroup$
    – venrey
    Commented Mar 22, 2019 at 21:28

1 Answer 1

Reset to default
1
$\begingroup$

For convenience, let $$\eqalign{ b &= w\odot G, \quad a &= 1_p \cr }$$ Rearrange the given differential to isolate the gradient wrt $Q$. $$\eqalign{ dP &= {\rm Tr}\Big(a^T\,(dQ\odot W)\,b\Big) \cr &= a^T\,(dQ\odot W)\,b \quad {\rm \{trace\,does\,not\,affect\,scalar\,values\}} \cr &= ab^T:(dQ\odot W) \cr &= (ab^T\odot W):dQ \cr \frac{\partial P}{\partial Q} &= ab^T\odot W \cr &= \Big(1_p(w\odot G)^T\Big)\odot W \cr }$$ where a colon is used to write the trace in product form, i.e. $$A:B = {\rm Tr}\big(A^TB\big)$$

UPDATE
The updated question uses $(f,g)$ in place of $(W,b)\,$ so the gradient becomes $$\eqalign{ \frac{\partial P}{\partial Q} &= ag^T\odot f \cr }$$

$\endgroup$
5
  • $\begingroup$ I apologize. Prior to my edit, I have written that $\mathbf{W}$ also involves $\mathbf{Q}$, i.e., it is not constant with respect to $\mathbf{Q}$. I have edited it now. $\endgroup$
    – venrey
    Commented Mar 23, 2019 at 1:10
  • $\begingroup$ Yes, but you also provided the differential $dP$ in which all those dependencies have been sorted out. The differential only contains the $dQ$ term. If that differential is correct, then the gradient above is correct. $\endgroup$
    – greg
    Commented Mar 23, 2019 at 1:27
  • $\begingroup$ But I can't seem to get pass with the third line of yours : $X = Q\odot W \implies dX = dQ\odot W$ where $X$ has just been introduced while the equality involving $dP$ has already been established prior to it. Did I miss something? Shouldn't that be product rule since $W$ involves $Q$? $\endgroup$
    – venrey
    Commented Mar 23, 2019 at 1:34
  • $\begingroup$ I changed the name of my variable from $P$ to $\phi$. The important point is that their differentials are equal. $\endgroup$
    – greg
    Commented Mar 23, 2019 at 1:37
  • $\begingroup$ The $X$ variable was a distraction. It has been removed. $\endgroup$
    – greg
    Commented Mar 23, 2019 at 18:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .