Use chain rule to compute partial derivative of $\sum\limits_k\left(\sum\limits_j\vert H_{kj}\vert\right)^2$ I am not sure how to separate function to apply chain rule for $\sum_{k}(\sum_{j}\vert H_{kj}\vert)^{2}$ wrt. to $H_{kj}$ where $H_{kj}$ is a matrix. What I have tried so far is:
$$g(h) = \sum_{j}\vert h_{j}\vert$$
$$f(x) = x^2$$
I omit $\sum_{k}$ first. Then, $(\sum_{j}\vert H_{kj}\vert)^{2}$ can be written as $f(g(H_{kj}))$:
\begin{equation}\begin{aligned} 
f(g(H_{kj}))' &= f'(g(H_{kj})) \times  g'(H_{kj}) \\
&= 2g(H_{kj}) \times \sum_{j}1 \\
&= 2\sum_{j}\vert H_{kj}\vert \times \sum_{j}1 \\
&= 2\sum_{j}\vert H_{kj}\vert \times j
 \end{aligned}\end{equation}
I think $\sum_{j}1$ should be sum of $1$, $j$ times but it seems weird. Is the final result $2j\sum_{kj}\vert H_{kj}\vert$ or $2\sum_{kj}\vert H_{kj}\vert$?
 A: Instead of the chain rule, you can solve the problem using differentials and standard matrix notations. 
For the elementwise/Hadamard and trace/Frobenius products we'll use the symbols
$$\eqalign{&A\odot B\cr &A:B}$$
 respectively. Note that $(A:B)$ is defined as $\,{\rm Tr}(A^TB)$
Next, define the elementwise sign function $\,\,S={\rm sign}(H)$
$$\eqalign{
S_{ij} &=
\begin{cases}
+1  &{\rm if\,\,\,\,}  H_{ij}>0 \\
 \,\,\,\,\,0  &{\rm if\,\,\,\,}  H_{ij}=0 \\
-1  &{\rm if\,\,\,\,}  H_{ij}<0 \\
\end{cases} \cr
}$$
The S-matrix allows us to write $\,\,\,|H|=S\odot H$
Finally, we'll use $u$ to denote the vector whose elements are all unity, so that summation over an index can be replaced by a matrix product with the $u$ vector, e.g. $\,\,Au = \sum_kA_{ik}$
In terms of these definitions, your function can be written in matrix notation 
$$\eqalign{
\phi &= u:\Big((S\odot H)u\Big)\odot\Big((S\odot H)u\Big) \cr
}$$
To reduce some clutter, let's temporarily define a vector $\,\,x=(S\odot H)u$
and find the differential and gradient of the function.
$$\eqalign{
\phi &= u:(x\odot x) \cr
d\phi &= u:(2x\odot dx) = 2x:dx = 2x:(S\odot dH)u = 2(S\odot xu^T):dH \cr
\frac{\partial\phi}{\partial H} &= 2S\odot xu^T
 = 2S\odot\Big((S\odot H)uu^T\Big) 
 = 2S\odot\Big(\,|H|\,uu^T\Big) \cr
\cr
}$$
Note that $uu^T$ is a square matrix whose elements are all unity.
