Derivative of a Matrix to a Power Fix a positive interger $k$ and let $F: \mathbb{R}^{n \times n} \rightarrow \mathbb{R}^{n \times n}$ be the map on $n \times n$ matrices defined by $F(A)= A^k$. Show that $F$ is differentiable at every point of $\mathbb{R}^{n \times n}$, and find $(DF)(A)$. 
So my first guess was the obvious one $(DF)(A)= k A^{k-1}$. So I looked at $$\frac{F(X) - F(A) - DF(X-A)}{|X-A|} = \frac{X^k - A^k - k (X-A)^{k-1}}{|X-A|}$$
And now  I don't know how to proceed. I not even sure if this is the right linear transformation for $DF$...
 A: I think you have a fundamental misconception about derivatives. The derivative of $F$ at $A$ is not a matrix in $\mathbb R^{n\times n}$, but a linear map from $\mathbb R^{n\times n}$ to $\mathbb R^{n\times n}$. (See, e.g. definition 2.1 on p.4 of this set of USC course notes.)
To show that $F$ is differentiable at every $A$, you only need to show that $(A+H)^k-A^k=L_A(H)+o(H)$, where $L_A:\mathbb R^{n\times n}\to\mathbb R^{n\times n}$, where both $L_A$ is a linear function in $H$ and the little-O constant for $o(H)$ depends on $n$ and $A$ but not $H$. The linear map $L_A$ is then the derivative $DF(A)$. Since $(A+H)^k-A^k$ is a matrix polynomial in $H$, $L_A(H)$ is actually the sum of all terms that are linear in $H$ and the $o(H)$ part essentially contains those higher-order terms in $H$.
To illustrate, suppose $k=2$. Then $(A+H)^2-A^2=AH+HA+H^2$. So $L_A(H)=AH+HA$, i.e. $DF(A)$ is the linear map $H\mapsto AH+HA$. Note that $H^2=o(H)$ because $\|H^2\|\le\|H\|^2=o(\|H\|)$ when a submultiplicative matrix norm is used.
You may obtain a matrix representation of $DF(A)$ if you want, but the matrix will live in $\mathbb R^{n^2\times n^2}$, not $\mathbb R^{n\times n}$, and such a matrix representation is not required in the question.
