Differentiation of a function defined over $\mathbb{R}^{m^{2}}$ (matrix spaces). 
For $1\leq i,j\leq m$, let $f_{ij}:\mathbb{R}^{m^{2}} \to \mathbb{R}$ defined by $f_{ij}(X)V = (X^{2})_{ij}$. For any $X,V \in \mathbb{R}^{m^{2}}$, show that $df_{ij}(X)V$ is the $ij$-th element of the matrix $XV + VX$. Get a similar result with $X^{3}$. Generalize.

I don't want an answer. Actually, I didn't understand how this function works. I would like an explanation to understand this function and a small hint to start solving.
 A: Recall the definition of the differential. $df_{ij}(X) : \mathbb{R}^{m^2} \to \mathbb{R}$ is the unique linear map such that
$$\lim_{H \to 0} \frac{\left|f_{ij}(X + H) - f_{ij}(X) - df_{ij}(X)H\right|}{\|H\|} = 0$$
where $\|\cdot\|$ is the euclidean norm on matrices.
It suffices to verify that $V \mapsto (XV + VX)_{ij}$ satisfies the above limit.
\begin{align}
\lim_{H \to 0} \frac{\left|f_{ij}(X + H) - f_{ij}(X) - (XH + HX)_{ij}\right|}{\|H\|} &= \lim_{H \to 0} \frac{\left|((X + H)^2)_{ij} - (X^2)_{ij} - (XH + HX)_{ij}\right|}{\|H\|}\\
 &= \lim_{H \to 0}\frac{\left|(X^2 + XH + HX + H^2)_{ij} - (X^2)_{ij} - (XH + HX)_{ij}\right|}{\|H\|}\\
&= \lim_{H\to 0} \frac{\left|(H^2)_{ij}\right|}{\|H\|}\\
&\le \lim_{H\to 0} \frac{\|H^2\|}{\|H\|}\\
&\le \lim_{H\to 0} \frac{\|H\|^2}{\|H\|}\\
&= \lim_{H\to 0} \|H\|\\
&= 0
\end{align}
Therefore $df_{ij}(X)V = (XV + VX)_{ij}$.
A: We can actually calculate $df_{ij}(X)\cdot V$. Remember that $df_{ij}(X)\cdot V=\langle\text{grad } f_{ij}(X),V\rangle=\dfrac{\partial f_{ij}}{\partial V}(X)$, then
\begin{align*}
df_{ij}(X)\cdot V&=\dfrac{\partial f_{ij}}{\partial V}(X)=\lim_{t\to0}\dfrac{f_{ij}(X+tV)-f_{ij}(X)}{t}\\&=\lim_{t\to0}\dfrac{(X^2+tXV+tVX+t^2V^2)_{ij}-(X^2)_{ij}}{t}\\
&=\lim_{t\to0}\dfrac{t(XV+VX)_{ij}+t^2(V^2)_{ij}}{t}=(XV+VX)_{ij}
\end{align*}
It's analogous for $f_{ij}(X)=(X^3)_{ij}$, you just need to observe that
$$([X+V]^3)_{ij}-(X^3)_{ij}=t(X^2V+XVX+VX^2)_{ij}+t^2(XV^2+VXV+tV^2)_{ij}$$
The geral answer can be found after suspecting for $g_k(X)=X^k$ we have $$dg_k(X)\cdot V=\displaystyle\sum_{i=1}^kX^{i-1}VX^{k-i}$$
Using induction :
\begin{align*}
dg_{k+1}(X)\cdot V=\dfrac{\partial g_{k+1}(X)}{\partial V}&=\dfrac{\partial(XX^k)}{\partial V}=\dfrac{\partial (Xg_k(X))}{\partial V}\\&=\dfrac{\partial X}{\partial V}\cdot g_k(X)+X\dfrac{\partial g_k(X)}{\partial V}\\&=VX^k+Xdg_k(X)\cdot V\\&=VX^k+X\displaystyle\sum_{i=1}^kX^{i-1}VX^{k-i}=\sum_{i=1}^{k+1}X^{i-1}VX^{k+1-i}\end{align*}
