Derivative of exponential function wrt a vector Let $\mathbf{A}$ be a $k\times k$ invertible matrix, let $\mathbf{x}$ be a $k\times 1$ vector and let $\mathbf{1}$ be a $k\times 1$ vector of ones. For a generic $k\times 1$ vector $\mathbf{z}$, let the function $\exp\left(\cdot\right)$  be defined as follows:
$\exp\left(\mathbf{z}\right)=\exp\left(\left[\begin{array}{c}
z_{1}\\
z_{2}\\
\vdots\\
z_{k}
\end{array}\right]\right)=\left[\begin{array}{c}
e^{z_{1}}\\
e^{z_{2}}\\
\vdots\\
e^{z_{k}}
\end{array}\right]$
Is the following equality true? If so, under what conditions?
$\frac{d}{d\mathbf{x}}\;\mathbf{1}'\mathbf{A}^{-1}\exp\left(\mathbf{Ax}\right)=\exp\left(\mathbf{Ax}\right)$
More in general, I am looking for a scalar function whose derivative with respect to vector $\mathbf{x}$ is $\exp\left(\mathbf{Ax}\right)$ (or its transpose).
Thanks a lot in advance!
 A: Define two new variables and their differentials
$$\eqalign{
y &= Ax &\implies dy=A\,dx \cr
e &= \exp(y) &\implies de = {\rm Diag}(e)\,dy = E\,dy \cr
}$$
Now finding the differential and gradient of your function is straightforward 
$$\eqalign{
 \phi &= 1:A^{-1}e \cr
d\phi
 &= 1:A^{-1}de \cr
 &= 1:A^{-1}E\,dy \cr
 &= 1:A^{-1}EA\,dx \cr
 &= A^TEA^{-T}1:dx \cr
\frac{\partial\phi}{\partial x}
 &= A^TEA^{-T}1 \cr
 &= A^T{\rm Diag}(e)A^{-T}1 \cr\cr
}$$
In some of the steps above, a colon is denote the trace/Frobenius product, i.e.
$$A:B = {\rm tr}(A^TB)$$
There are many ways to rearrange the arguments in a Frobenius product, which follow from the cyclic properties of the trace function.
For example, all of the following are equivalent
$$\eqalign{
  A:BC
 &= BC:A \cr
 &= A^T:(BC)^T \cr
 &= B^TA:C \cr
 &= AC^T:B \cr\cr
}$$
For your general question, notice that if we use an invertible diagonal matrix
$$A={\rm Diag}(a)$$
then (since diagonal matrices commute) the above result reduces to
$$\frac{\partial\phi}{\partial x} = E1 = \exp(Ax) $$
A: OK, I'll give you a counterexample that your quest is impossible for a 2x2 matrix, for simplicity; that is, you assume there is a potential φ (pardon the physicsese...), s.t.
$$\frac{d}{d\mathbf{x}} \phi =\exp\left(\mathbf{Ax}\right) \qquad \Longrightarrow \\
\frac{d}{d x _1} \phi =\exp\left(A_{11}x_1+A_{12}x_2\right) , \qquad \frac{d}{d x _2} \phi =\exp\left(A_{21}x_1+A_{22}x_2\right) .
$$
Now the integrability condition for this simplest linear system is, by above, 
$$
\left (\frac{d}{d x _2}  \frac{d}{d x _1} - \frac{d}{d x _1}  \frac{d}{d x _2}\right) \phi=0=A_{12}\exp\left(A_{11}x_1+A_{12}x_2\right)    - A_{21}  \exp\left(A_{21}x_1+A_{22}x_2\right) .
$$
That is,
$$
A_{12}e^{\left(A_{11}-A_{21}\right)x_1 }   = A_{21}   e^{\left(A_{22} -A_{12}\right)x_2},
$$
hopeless except in special circumstances, such as diagonal A, as  proffered in @greg 's answer.
