Derivative of an exponential matrix-based function I have a linear matrix operator $A$ and the vector $x$. A function of $x$ is given by
$$
F = Ax^n,
$$
where $x^n$ represents element-wise power.
I calculate $\frac{dF}{dx}$ as $\frac{dF}{dx} = A\times \operatorname{diag}(nx^{n-1})$ and this works.
Consider the following function -
$E = \exp(F)$ where $exp(.)$ represents element-wise exponential. I would like to calculate $\frac{dE}{dx}$.
My idea was to do it as: $\frac{dE}{dx} = \exp(F) \times F'$ where $F'$ has been calculated above. But the vector times the matrix doesn't seem to be possible in this context and interchanging the terms doesn't work either.
How do I calculate $\frac{dE}{dx}$?
 A: The chain rule says that $$\dfrac{dE}{dx}=(\exp'\circ F)\cdot F'.$$  However, it is not correct to simplify $\exp'$ down to $\exp$ in this context, since $\exp$ is not just the ordinary one-variable exponentiation function.  Instead, $\exp$ is the function $\mathbb{R}^n\to\mathbb{R}^n$ given by $\exp(x_1,\dots,x_n)=(e^{x_1},\dots,e^{x_n})$.  The derivative of this function at a point $(x_1,\dots,x_n)$ is the $n\times n$ diagonal matrix whose diagonal entries are $e^{x_1},\dots,e^{x_n}$ (or to use your earlier notation, you could call it $\operatorname{diag}(\exp)$).  So $\exp'\circ F$ is the diagonal matrix whose diagonal entries are given by exponentiating the coordinates of $F$ (or $\operatorname{diag}(\exp(F))$ in your notation).
A: $
\def\l{\left}
\def\r{\right}
\def\o{{\large\tt1}}
\def\p{\partial}
\def\lr#1{\l(#1\r)}
\def\diag#1{\operatorname{diag}\lr{#1}}
\def\Diag#1{\operatorname{Diag}\lr{#1}}
\def\grad#1#2{\frac{\p #1}{\p #2}}
\def\bgrad#1#2{\lr{\frac{\p #1}{\p #2}}}
\def\c#1{\color{red}{#1}}
$Let's use a convention in which uppercase letters denote matrices and lowercase letters vectors.
Further, if a matrix has the same name as a vector then it represents the diagonal matrix generated by that vector, e.g.
$$\eqalign{
&A = \Diag{a}, \quad B = \Diag{b}, \quad etc \\
&a = A\o, \quad b = B\o, \quad \ldots \\
}$$ where $\o$ is the all-ones vector.
The nice thing about diagonal matrices is that they commute, which makes them almost as easy to work with as scalars. Here are some examples
$$\eqalign{
\grad{A^n}{z} &= nA^{n-1}\bgrad{A}{z} \\
AB &= BA \\
a\odot b &= Ab = AB\o = BA\o = Ba = b\odot a \\
}$$ where $(\odot)$ denotes the elementwise/Hadamard product.
In the current problem, we have
$$\eqalign{
f &= Ax^{\odot n} = AX^n\o \\
df &= nAX^{n-1}\,dX\,\o = nAX^{n-1}\,dx \\
\grad{f}{x} &= nAX^{n-1} \\
}$$
Continuing to the exponential function
$$\eqalign{
p &= \exp(f) \\
dp &= p\odot df = P\,df \\
\grad{p}{x} &= P \bgrad{f}{x} = nPAX^{n-1} \\
}$$
A: For square matrices the matrix exponential works. Normally this consists of writing the matrix in the form:
$$M=PDP^{-1}$$
such that:
$$\sum\frac{M^n}{n!}=P\left(\sum\frac{D^n}{n!}\right)P^{-1}$$

In terms of the exponential of a vector, It only makes sense if it is termwise, so once again I would use the expansion for the exponential and sum them up for each term
