# Derivative of vector valued function with Kronecker products involved

I would like to calulate the derivative of the following function $$f:R^m \rightarrow R$$ with: $$f(\mathbf{x})=f(x_1,\dots,x_m )= \| A - G B\|_F^2$$, where $$A \in C^{n\times k},$$ $$B \in C^{m\times k},$$ and $$G \in C^{n\times m}$$ with $$G:=[\mathbf{g}(x_1),\dots,\mathbf{g}(x_m)]$$, where $$\mathbf{g}:R\rightarrow C^n$$ is a differentiable function with derivative $$\mathbf{g}'(x)$$.

If we express the inner product in vector form and define $$W=A - G B$$ we have:

$$f(\mathbf{x})=\mathrm{vec}(W)^H\mathrm{vec}(W)$$. Hence, its differential is:

$$df = 2\mathrm{Re}\{ \mathbf{w^*}:d \mathbf{w}\},$$ where $$\mathbf{w}=\mathrm{vec}(W).$$

Next, follows: $$d \mathbf{w}=-(B^T \otimes I_n)d \mathbf{g}$$, where $$\mathbf{g} = \mathrm{vec}(G).$$ Now for $$d \mathbf{g}$$ I came to the following expression:

$$d \mathbf{g}= d \begin{bmatrix}\mathbf{g}(x_1)\\ \vdots\\ \mathbf{g}(x_m)\end{bmatrix}= \begin{bmatrix}\mathbf{g}'(x_1)dx_1\\ \vdots\\ \mathbf{g}'(x_m)dx_m \end{bmatrix}=\mathrm{vec}(H \mathrm{diag}(d\mathbf{x})) = (I_m \otimes H) \mathrm{vec}(\mathrm{diag}(d\mathbf{x}))$$, where

$$H=\begin{bmatrix}\mathbf{g}'(x_1), \dots, \mathbf{g}'(x_m)\end{bmatrix}$$ is a $$n\times m$$ matrix of derivatives. Finally, in my attempt to derive the gradient I came to:

$$\mathbf{w}^*:d\mathbf{w}=- (I_m \otimes H^T)(B \otimes I_n)\mathbf{w}^*:\mathrm{vec}(\mathrm{diag}(d\mathbf{x})).$$

However, I am stuck and not sure I can derive the gradient from that formula. Can I express the vector on the right with a selection matrix from a higher dimension?

Any help will be greatly appreciated.

First, as you have done, define the matrices \eqalign{ W &= GB-A,\quad X={\rm Diag}(x),\quad x={\rm diag}(X) \cr G &= g(x),\quad H = g'(x) \quad\implies\; dG = H\,dX \cr X &= X^H = X^T = X^* \cr } Write the function in terms of these matrices.
Then calculate its differential and gradient. \eqalign{ f &= W^*:W \cr df &= W^*:dW \;+\; W:dW^* \cr &= W^*:dG\,B \;+\; W:dG^*B^* \cr &= W^*B^T:dG \;+\; WB^H:dG^* \cr &= W^*B^T:H\,dX \;+\; WB^H:H^*dX \cr &= H^TW^*B^T:dX \;+\; H^HWB^H:dX \cr &= (H^TW^*B^T + H^HWB^H):{\rm Diag}(dx) \cr &= {\rm diag}(H^TW^*B^T + H^HWB^H):dx \cr \frac{\partial f}{\partial x} &= {\rm diag}(H^TW^*B^T + H^HWB^H) \cr &= 2\,{\cal Re}\Big({\rm diag}(H^TW^*B^T)\Big) \cr &= 2\,{\cal Re}\Big({\rm diag}(H^HWB^H)\Big) \cr &= 2\,{\cal Re}\Big({\rm diag}(B^*W^TH^*)\Big) \cr &= 2\,{\cal Re}\Big({\rm diag}(BW^HH)\Big) \cr } Note that a colon indicates the trace/Frobenius product, i.e. \eqalign{ A:B = {\rm Tr}(A^TB)} Also note that my $$W$$ matrix is defined as the negative of yours.