Gradient of the matrix exponential I need to solve a simple optimization problem, and I would like to use gradient descent.
Let $O \in \mathbb{R}^{n\times n}$ a known orthogonal matrix with determinant $1$. Consider the following cost function
$$
\mathcal{L}(X) = \|e^X - O\|_2
$$
and the problem of minimizing $\mathcal{L}(X)$ under the constraint that $X$ is skew-symmetric:
$$
\min\,\mathcal{L}(X),\,\,\,s.t.\,\,\,X^\top = −X
$$
Assuming to use gradient descent to solve the problem, I would need to calculate the gradient of $\mathcal{L}(X)$ with respect to the entries of $X$, a problem that requires the knowledge of the following derivatives:
$$
\frac{\partial e^X}{\partial X_{i,j}}
$$
Is there any known formula to calculate these derivatives?
 A: There is a nice formula for this derivative, in terms of the exponential and the adjoint action $\mathrm{ad}_X$, defined so that $\mathrm{ad}_X Y=[X, Y]=XY-YX$.
$$
\frac{\partial}{\partial\alpha} \exp{X} = \exp{X}\frac{1-\exp\left(-\mathrm{ad}_X\right)}{\mathrm{ad}_X}\frac{\partial}{\partial\alpha} X
$$
See this wikipedia page for reference. For $n=2$ or $n=3$, this has a simple closed form; for $n=2$, for instance, $\mathrm{ad}_X=0$ and $\frac{\partial}{\partial\alpha} \exp{X} = \exp{X}\frac{\partial}{\partial\alpha} X$, and for $n=3$, you can derive (after a fair bit of algebra) that if 
$$X=\mathrm{sk}(x)=\begin{pmatrix} 0 & x_3 & -x_2 \\ -x_3 & 0 & x_1 \\ x_2 & -x_1 & 0 \end{pmatrix},$$
then
$$
\frac{\partial}{\partial\alpha}\exp(X) = \mathrm{sk}\left(\left(\mathbf{1} + \frac{1-\cos\Vert x\Vert}{\Vert x\Vert^2}X + \frac{\Vert x\Vert-\sin\Vert x\Vert}{\Vert x \Vert^3}X^2\right)\frac{\partial x}{\partial\alpha}\right)\exp(X).$$ There may be comparably simple forms for larger $n$ as well.
However, I'm also fairly sure that unless there are other unstated constraints, the solution to your optimization problem is $\exp X = O$, since every special orthogonal matrix can be represented as the exponential of a skew-symmetric matrix.
