As for analytical functions in one dimension, one can define a matrix function
for square matrices $\mathbf{X}$ by an infinite series:
$$ f(\mathbf{X}) = \sum_{n=0}^\infty c_n \mathbf{X}^n$$ assuming the limit exists and is finite. If the coefficients $c_n$ fulfils $\sum c_n x^n <\infty$,
then one can prove that the above series exists and is finite
source: The matrix cookbook
So we know that $\exp(t\mathbf{A})$:
$$ \exp(t\mathbf{A}) = \sum_{n=0}^\infty \frac{t^n \mathbf{A}^n}{n!} $$
and thus
$$ \frac{\textrm{d}}{\textrm{d}t}\exp(t\mathbf{A}) = \sum_{n=1}^\infty \frac{n t^{n-1} \mathbf{A}^n}{n!} = \sum_{n=0}^\infty \frac{t^{n} \mathbf{A}^{n+1}}{n!}=\textbf{A}\,\exp(t\mathbf{A}) = \exp(t\mathbf{A})\,\textbf{A} $$
The OP is interested in
$$ \frac{\partial}{\partial t_p}\exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\}. $$
This is not as straightforward as it seems. To explain this you need to understand that:
$$ \exp(\mathbf{A} + \mathbf{B}) = \exp(t\mathbf{A})\,\exp(t\mathbf{B}),\quad\textrm{if}\quad \mathbf{AB}=\mathbf{BA} $$
If $\mathbf{A}$ and $\mathbf{B}$ do not commute, this is not the case. You can see this from the simple expansion:
$$ \exp(\mathbf{A} + \mathbf{B}) = \sum_{n=0}^\infty \frac{t^n (\mathbf{A}+\mathbf{B})^n}{n!}$$
Only if $\mathbf{A}$ and $\mathbf{B}$ commute, you can use the binomial expansion, in the other case it becomes really messy. To give an example, the second term becomes:
$$\begin{align}
\exp(\mathbf{A} + \mathbf{B}) =& \mathbf{I} + \mathbf{A}+\mathbf{B} \\
& + \frac{1}{2}\left(\mathbf{A}^2 + \mathbf{AB} + \mathbf{BA} + \mathbf{B}^2\right) \\
& + \frac{1}{3!}\left(\mathbf{A}^3 + \mathbf{A}^2\mathbf{B} + \mathbf{A}\mathbf{B}\mathbf{A} +\mathbf{B}\mathbf{A}^2 + \mathbf{B}^2\mathbf{A} + \mathbf{B}\mathbf{A}\mathbf{B} +\mathbf{A}\mathbf{B}^2+ \mathbf{B}^3\right)\\
&+\ldots
\end{align}$$
So, in the OP, if $\mathbf{A}_p\mathbf{A}_q = \mathbf{A}_q\mathbf{A}_p$ for all combinations of $p$ and $q$, then
$$\frac{\partial}{\partial t_p}\exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\} = \textbf{A}_p\,\exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\} = \exp\left\{\sum_{i=1}^nt_i\mathbf{A}_i\right\}\,\textbf{A}_p$$
If the matrices do not commute, i.e. $\mathbf{A}_p\mathbf{A}_q \ne \mathbf{A}_q\mathbf{A}_p$ for any combination of $p$ and $q$, it quickly blows up.