Differentiating a function of matrices Suppose I have a function $\phi(M):=M^n$ where $M\in M_n(R)$
Does it make any sense to talk about the derivative of this function? What would it mean if it does make sense? I am also wondering what the derivative (if it exists and is meaningful) would be. (Perhaps $nM^{n-1}$?)
Thanks.
 A: Yes, it makes sense to talk about derivatives of functions $f:V\to W$, where $V$ and $W$ are a normed vector space, for example. 
In this case, we say that $f$ is differentiable at $x\in V$ if there is a linear mapping $Df(x):V\to V$ such that 
$$
\lim_{h\to 0}\frac{\|f(x+h)-f(x)-Df(x)h\|}{\|h\|}=0
$$
In your particular case, you could compute the derivative of $M\mapsto M^n$ 
by using a $n$-linear application given by
$$
\varphi(A_1,\ldots,A_n)=A_1\cdot\ldots\cdot A_n.
$$
You can show for any $X=(X_1,\ldots,X_n)\in M_n(R)\times\ldots\times M_n(R)$ ($n$-times) that 
$$
D\varphi(X)(H_1,\ldots,H_n)=\sum_{i=1}^n \varphi(X_1,\ldots,X_{i-1},H_i,X_{i+1},\ldots,X_n).
$$ 
So the formula you want should be interpreted as the derivative of $\varphi$ at $X=(M,\ldots,M)$ applied at $(Id,\ldots,Id)$. Which is equivalent to what we do when we write down this formula for real and complex functions.
A: Layman version:
Start with $n=1$: $f(M) = M$. Then $Df(M) = DM = I$.
For $n=2$: $f(M) = M*M$. Apply product rule: $$Df(M) = (DM) * M + M * (DM) = I*M + M*I = 2M$$
Induction on general $n\geq2$, assuming $D(M^{n-1}) = (n-1)*M^{n-2}$: $f(M) = M*M^{n-1}$. Product rule: $$Df(M) = (DM) * M^{n-1} + M * D(M^{n-1}).$$ Use induction:
 $$D(M^{n-1}) = (n-1)M^{n-2}.$$ Then $$Df(M) = I * M^{n-1} + M*(n-1)*M^{n-2} = n*M^{n-1}.$$
QED.
Of course, I'm assuming that the product rule holds.
