Derivative of matrix exponential of linear combination Let's say $t$ is a real parameter and $\textbf{A}$ is $n \times n$ matrix. I know that
$$\frac{d}{dt} \exp(t\textbf{A}) = \textbf{A} \exp(t\textbf{A})$$
But what if there are multiple parameters $t_1, ..., t_n$ and multiple matrices $\textbf{A}_1, \dots , \textbf{A}_n$ that don't commute, does the same relation, namely
$$\frac{\partial}{\partial t_k}\exp \left(\sum_{i=1}^n t_i \textbf{A}_i \right) = \textbf{A}_k \exp \left(\sum_{i=1}^n t_i \textbf{A}_i\right)$$
still hold? If not, is there a closed form for the derivative?
 A: PARTIAL ANSWER. 
If there is a general formula for 
$$
\partial_t e^{tX+sY}, \qquad \partial_s e^{tX+sY}, $$ 
then it surely descends from the Baker-Campbell-Hausdorff formula, which is known to be rather complicated. So, I will only consider a simpler special case.

In the special case of $X$ and $Y$ commuting with the commutators, that is $$\tag{1}[X, [X, Y]]=[Y, [X, Y]]=0,$$ 
the Baker-Campbell-Hausdorff formula simplifies to
$$\tag{2}
e^{tX+sY+\frac{ts}{2}[X, Y]}=e^{tX}e^{sY}, $$
and since, by (1), $tX+sY$ commutes with $[X, Y]$,
$$
e^{tX+sY}=e^{tX}e^{sY}e^{-\frac{ts}{2}[X, Y]}, $$
from which we infer
$$\begin{split}
\partial_t e^{tX+sY} &= Xe^{tX}e^{sY}e^{-\frac{ts}{2}[X,Y]} - e^{tX}e^{sY}\left(\frac{s}{2}[X, Y]\right)e^{-\frac{ts}{2}[X,Y]}\\
&=(X-\frac{s}{2}[X, Y])e^{tX+sY},
\end{split}$$
where we used (1), which implies that $[X,Y]$ commutes with $e^{tX}$ and with $e^{sY}$.
Reversing the roles of $X$ and $Y$, and using that $[Y, X]=-[X, Y]$, we have 
$$
\partial_s e^{tX+sY} = (Y+\frac{t}{2}[X, Y])e^{tX+sY}.$$
In particular, we see that in general it is not true that 
$$\tag{!!}
\partial_t e^{tX+sY} = X e^{tX+sY}.$$ 

SOME WORDS ON THE GENERAL CASE. 
If we drop the assumption (1), then the Baker-Campbell-Hausdorff formula becomes much more complicated; 
$$\tag{3}
e^{tX+sY+\frac{ts}{2}[X, Y]+\frac1{12}\left( t^2s[X,[X,Y]] - ts^2[Y, [X,Y]]\right)+\ldots} =e^{tX}e^{sY}.$$
I don't know how to pass from (3) to a formula of the form 
$$
e^{tX+sY}=e^{tX}e^{sY}e^{\Phi(t, s)},$$ 
which is what we need for the above computation.
A: 
As for analytical functions in one dimension, one can define a matrix function
  for square matrices $\mathbf{X}$ by an infinite series:
  $$ f(\mathbf{X}) = \sum_{n=0}^\infty c_n \mathbf{X}^n$$ assuming the limit exists and is finite. If the coefficients $c_n$ fulfils $\sum c_n x^n <\infty$,
  then one can prove that the above series exists and is finite
source: The matrix cookbook

So we know that $\exp(t\mathbf{A})$:
$$ \exp(t\mathbf{A}) = \sum_{n=0}^\infty \frac{t^n \mathbf{A}^n}{n!} $$
and thus
$$ \frac{\textrm{d}}{\textrm{d}t}\exp(t\mathbf{A}) = \sum_{n=1}^\infty \frac{n t^{n-1} \mathbf{A}^n}{n!} = \sum_{n=0}^\infty \frac{t^{n} \mathbf{A}^{n+1}}{n!}=\textbf{A}\,\exp(t\mathbf{A}) = \exp(t\mathbf{A})\,\textbf{A}  $$
The OP is interested in 
$$ \frac{\partial}{\partial t_p}\exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\}. $$
This is not as straightforward as it seems. To explain this you need to understand that:
$$ \exp(\mathbf{A} + \mathbf{B}) = \exp(t\mathbf{A})\,\exp(t\mathbf{B}),\quad\textrm{if}\quad \mathbf{AB}=\mathbf{BA} $$
If $\mathbf{A}$ and $\mathbf{B}$ do not commute, this is not the case. You can see this from the simple expansion:
$$ \exp(\mathbf{A} + \mathbf{B}) = \sum_{n=0}^\infty \frac{t^n (\mathbf{A}+\mathbf{B})^n}{n!}$$
Only if $\mathbf{A}$ and $\mathbf{B}$ commute, you can use the binomial expansion, in the other case it becomes really messy. To give an example, the second term becomes:
$$\begin{align}
\exp(\mathbf{A} + \mathbf{B}) =& \mathbf{I} + \mathbf{A}+\mathbf{B} \\
& + \frac{1}{2}\left(\mathbf{A}^2 + \mathbf{AB} + \mathbf{BA} + \mathbf{B}^2\right) \\
& + \frac{1}{3!}\left(\mathbf{A}^3 + \mathbf{A}^2\mathbf{B} + \mathbf{A}\mathbf{B}\mathbf{A} +\mathbf{B}\mathbf{A}^2 + \mathbf{B}^2\mathbf{A} + \mathbf{B}\mathbf{A}\mathbf{B} +\mathbf{A}\mathbf{B}^2+ \mathbf{B}^3\right)\\
&+\ldots
\end{align}$$
So, in the OP, if $\mathbf{A}_p\mathbf{A}_q = \mathbf{A}_q\mathbf{A}_p$ for all combinations of $p$ and $q$, then
$$\frac{\partial}{\partial t_p}\exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\} = \textbf{A}_p\,\exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\} = \exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\}\,\textbf{A}_p$$ 
If the matrices do not commute, i.e. $\mathbf{A}_p\mathbf{A}_q \ne \mathbf{A}_q\mathbf{A}_p$ for any combination of $p$ and $q$, it quickly blows up.
A: Let 
$$\eqalign{
t&=t_i,\quad A=A_i,\quad B=\sum_{k\ne i} t_kA_k \\
G(t) &= \frac{d}{dt}\exp(B+tA) \\
}$$
To evaluate $G(t)$ at $t=0$, use the block-triangular method.
$$\eqalign{
\exp\left(\left[\matrix{B&A\\0&B}\right]\right)
 &= \left[\matrix{\exp(B)&G(0)\\0&\exp(B)}\right] \\
}$$
To evaluate $G$ at $t=s,\,$ shift the definition of $B\to (B+sA)$
Define block-analogs of the standard basis vectors
$$e_1=\pmatrix{1\\0},\quad e_2=\pmatrix{0\\1}$$
by replacing the {${0,1}$} elements with the $n\times n\,$ {zero, identity} matrices
$$E_1=\pmatrix{I\\0},\quad E_2=\pmatrix{0\\I} \in{\mathbb R}^{2n\times n}$$
Then would this qualify as a closed-form solution?
$$\eqalign{
G(s) &= E_1^T\,\exp\left(\left[\matrix{B+sA&A\\0&B+sA}\right]\right)\,E_2 \\
}$$
