Why it is called the exponential
The reason it's called the exponential is that in the case of matrix manifolds,
the abstract version of $\exp$ defined in terms of the manifold structure coincides
with the "matrix exponential" $exp(M) \equiv \sum_{i=0}^\infty M^n/n!$.
A concrete example, the unit circle
For example, let's consider the unit circle $M \equiv \{ x \in \mathbb R^2 : |x| = 1 \}$.
This can be viewed as a Lie group
$M = G = SO(2) = \left\{ \begin{bmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{bmatrix} : \theta \in \mathbb R \right\}$.
The unit circle: Tangent space at the identity, the hard way
We can derive the lie algebra $\mathfrak g$ of this Lie group $G$ of this "formally"
by trying computing the tangent space of identity. To do this, we first need a
useful definition of the tangent space. One possible definition is to use
the definition of the space of curves $\gamma_{\alpha}: [-1, 1] \rightarrow M$, where
the curves are such that $\gamma(0) = I$. Then the
tangent space $T_I G$ is the collection of the curve derivatives $\frac{d(\gamma(t)) }{dt}|_0$.
Let's calculate the tangent space of $G$ at the identity matrix $I$, $T_I G$:
$$
\gamma_\alpha(t) =
\begin{bmatrix}
\cos (\alpha t) & \sin (\alpha t) \\
-\sin (\alpha t) & \cos (\alpha t)
\end{bmatrix}
$$
This is a legal curve because the image of $\gamma$ is in $G$, and $\gamma(0) = I$. We can
differentiate this and compute $d/dt(\gamma_\alpha(t))|_0$ to get:
\begin{align*}
&\frac{d/dt} \gamma_\alpha(t)|_0 =
\frac{d}{dt}
\begin{bmatrix}
\cos (\alpha t) & \sin (\alpha t) \\
-\sin (\alpha t) & \cos (\alpha t)
\end{bmatrix}|_0 \\
&=
\begin{bmatrix}
\frac{d(\cos (\alpha t))}{dt}|_0 & \frac{d(\sin (\alpha t))}{dt}|_0 \\
\frac{d(-\sin (\alpha t))}{dt}|_0 & \frac{d(\cos (\alpha t))}{dt}|_0
\end{bmatrix} \\
&=
\begin{bmatrix}
-t\sin (\alpha t)|_0 & t\cos (\alpha t)|_0 \\
-t\cos (\alpha t)|_0 & -t\sin (\alpha t)|_0
\end{bmatrix} \\
&=
\begin{bmatrix}
0 & t \cdot 1 \\
-t \cdot 1 & 0
\end{bmatrix} \\
= \text{skew symmetric matrix}
\end{align*}
So we get that the tangent space at the identity $T_I G = \{ S \text{ is $2\times2$ matrix} : S + S^T = 0 \}$
Now recall that the Lie algebra $\mathfrak g$ of a Lie group $G$ is
defined to be the tangent space at the identity. So we have that
$\mathfrak g = T_I G = \text{$2\times2$ skew symmetric matrices}$.
The unit circle: Tangent space at the identity by logarithmization
On the other hand, we can also compute the Lie algebra $\mathfrak g$ / the tangent
space at the identity $T_I G$ "completely informally",
by "logarithmizing" the group. We know that the group of rotations $SO(2)$ consists
of orthogonal matrices
group, so every element $U \in G$ satisfies $UU^T = I$. We can logarithmize this
condition as follows:
$$
M = G = \{ U : U U^T = I \} \\
\mathfrak g = \log G = \{ \log U : \log (U U^T) = \log I \} \\
\mathfrak g = \log G = \{ \log U : \log (U) + \log(U^T) = 0 \} \\
\mathfrak g = \log G = \{ \log U : \log (U) + \log(U)^T = 0 \} \\
\mathfrak g = \log G = \{ S : S + S^T = 0 \} \\
$$
We got the same result: $\mathfrak g$ is the group of skew-symmetric matrices by
following the physicist derivation of taking a $\log$ of the group elements.
The unit circle: The exponential map
Now, it should be intuitively clear that if we got from $G$ to $\mathfrak g$
using $\log$, we ought to have an nverse $\exp: \mathfrak g \rightarrow G$ which
does the opposite. Indeed, this is exactly what it means to have an exponential
map: we can go from elements of the Lie algebra $\mathfrak g$ / the tangent space
at the identity $T_I G$ to the Lie group $G$.
The unit circle: Computing the exponential map
Assume we have a $2 \times 2$ skew-symmetric matrix $S$. We want to show that its
exponential lies in $G$:
$$
\exp(S) = \exp \left (\begin{bmatrix} 0 & s \\ -s & 0 \end{bmatrix} \right) =
\sum_{n=0}^\infty S^n/n!
$$
We can compute this by making the following observation:
\begin{align*}
S^2 =
\begin{bmatrix}
0 & s \\ -s & 0
\end{bmatrix}
\begin{bmatrix}
0 & s \\ -s & 0
\end{bmatrix}
= \begin{bmatrix}
-s^2 & 0 \\ 0 & -s^2
\end{bmatrix}
= -\begin{bmatrix}
s^2 & 0 \\ 0 & s^2
\end{bmatrix}
\end{align*}
We immediately generalize, to get $S^{2n} = -(1)^n
\begin{bmatrix}
s^{2n} & 0 \\ 0 & s^{2n}
\end{bmatrix}$
This gives us $S^{2n+1} = S^{2n}S$:
\begin{align*}
S^{2n+1} = S^{2n}S =
(-1)^n
\begin{bmatrix}
s^{2n} & 0 \\ 0 & s^{2n}
\end{bmatrix}
\begin{bmatrix}
0 & s \\ -s & 0
\end{bmatrix}
=
(-1)^n
\begin{bmatrix}
0 & s^{2n+1} \\ -s^{2n+1} & 0
\end{bmatrix}
\end{align*}
We can now compute the exponential as:
\begin{align*}
&\exp(S) = I + S + S^2 + S^3 + .. = \\
&(I + S^2/2! + S^4/4! + \cdots) + (S + S^3/3! + S^5/5! + \cdots) \\
&= \begin{bmatrix}
1 - s^2/2! + s^4/4! + \cdots & 0 \\
0 & 1 - s^2/2! + s^4/4! + \cdots
\end{bmatrix} +
\begin{bmatrix}
0 & s - s^3/3! + s^5/5! + \cdots \\
s - s^3/3! + s^5/5! + \cdots & 0
\end{bmatrix} \\
&=
\begin{bmatrix}
\cos(s) & \sin(s) \\
-sin(s) & \cos(s)
\end{bmatrix}
\end{align*}
We get the result that we expect: We get a rotation matrix $\exp(S) \in SO(2)$.
We can check that this $\exp$ is indeed an inverse to $\log$.
Why skew-symmetric?
What does it mean that the tangent space at the identity $T_I G$ of the
group of rotations are the skew-symmetric matrices?
One explanation is to think of these as curl, where a curl is a sort
of "infinitesimal rotation". See that a skew symmetric matrix
$S \equiv \begin{bmatrix}
a & b \\ -b & a
\end{bmatrix}$
can be viewed as having two vectors $S_1 = (a, b)$ and $S_2 = (-b, a)$, which
represents an infinitesimal rotation from $(a, b)$ to $(-b, a)$.
This is skew-symmetric because rotations in 2D have an orientation. Flipping
the order of the vectors gives us the rotations in the opposite order: It takes
clockwise to anti-clockwise and anti-clockwise to clockwise. If we wish
to fancy, we can talk about this in terms of exterior algebra
See the picture which shows the skew-symmetric matrix $\begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}$ and its transpose as "2D orientations"
The unit circle: What about the other tangent spaces?!
So far, I've only spoken about the lie algebra $\mathfrak g$ / the tangent space at
the identity $T_I G$. What about all of the other tangent spaces?
A very cool theorem of matrix Lie theory tells
us that the tangent space at some point $P$, $T_P G$ is always going
to be translates of $T_I G$. Formally, we have the equality:
$$T_P G = P T_I G = \{ P T : T \in T_I G \}$$
This lets us immediately know that whatever theory we have discussed "at the identity"
can be easily translated to "any point" $P \in G$, by simply multiplying with the point $P$.
Recap
We saw the following equivalences:
- We have a Lie group $G$ with Lie algebra $\mathfrak g$, which is the
tangent space at the identity $T_I G$.
- For this, computing the Lie algebra by using the "curves" definition co-incides
with simply invoking $\log: G \rightarrow \mathfrak g$ on the definiton of the matrix group.
- Vice versa, the $\exp$ (inverse of $\log$) can be computed from the series
definition, giving us a map $\exp: \mathfrak g \rightarrow G$.
- These maps allow us to go from the "local behaviour" to the "global behaviour".
- We gained an intuition for the concrete case of $G = SO(2)$, $\mathfrak g$ as
skew-symmetric matrices, and why skew-symmetric matrices are the "infinitesimal rotations"
- We refer to the fact that if we know $\mathfrak g$/$T_I G$, we automatically know
all tangent spaces due to the group being a Lie group. Hence, knowing the lie
algebra $\mathfrak g$ is "as good as" knowing the tangent space structure everywhere.
A summary picture:
