Is there a general formula for the derivative of $\exp(A(x))$ when $A(x)$ is a matrix? It's easy for scalars, $(\exp(a(x)))' = a' e^a$. But can anything be said about matrices? Do $A(x)$ and $A'(x)$ commute such that $(\exp(A(x)))' = A' e^A = e^A A'$ or is this only a special case?
 A: The question comes down to computing what's called the "derivative of", the "differential of", or the "tangent map to", the exponential map from $M_n(\mathbb R)$ into itself at a given matrix $A$ (not necessarily the zero matrix). There is a classical formula for this. Here is the first reference I found: pages 1 and 2 of
http://www.math.columbia.edu/~woit/notes4.pdf
by Peter Woit. Here it is (with Peter Woit's notation) 
$$\exp_*(X)\ Y=\exp(X)\ \frac{1-e^{-ad(X)}}{ad(X)}\ Y.$$ 
Here is a reference for the Chain Rule:
http://en.wikipedia.org/wiki/Chain_rule#The_fundamental_chain_rule
It reads, in Peter Woit's notation and under appropriate assumptions,
$$(f\circ g)\_*(x)=f_*(g(x))\circ g_*(x).$$
[Thank you to KennyTM for having edited this formula.]  
EDIT 1. Here are the two formulas written in another notation:
$$\exp'(X)=\exp(X)\ \frac{1-e^{-ad(X)}}{ad(X)}\quad,$$ 
$$(f\circ g)'(x)=f'(g(x))\circ g'(x).$$
EDIT 2. Here is another reference. This is a post by Akhil Mathew:
http://deltaepsilons.wordpress.com/2009/11/07/helgasons-formula-ii/
A: Tom Stephens linked to the exponential map, which states that

$ \frac{d}{dt}e^{X(t)} = \int\limits_0^1 e^{\alpha X(t)} \frac{dX(t)}{dt} e^{(1-\alpha) X(t)} d\alpha $

If $X(t)$ and $\frac{d}{dt}X(t)$ commute, the latter also commutes with $\exp(X(t))$ and then it simplifies into $ \frac{d}{dt}e^{X(t)} = \frac{d X(t)}{dt} e^{X(t)}$.
A counter-example is
$$X(t) = \begin{pmatrix} \cos(t) & \sin(t) \\\ \sin(t) & -\cos(t) \end{pmatrix}$$
at $t=0$, i.e. $X(0) = \sigma_3, \dot X(0) = \sigma_1$ ($\sigma_i$ are the non-commuting Pauli Matrices)
A: Only I khnow is that if your matrix $A$ and $A\in GL(n,\mathbb{R})$ is invertible we can say that $A'(X)=-AXA^{-1}$.
So $(e^{A(x)})' =A(x)'e^ {A(x)} =-AXA^{-1}e^{A(x)}$ , since $A$ is not commutative you can't say $A' e^{A(x)} =e^{A(x)}A'$
