Gauss Lemma - Do Carmo's Riemannian geometry, use of parallel transport? I was exactly having the same doubt as this question. I don't understand specifically why
$$
(d \exp_p)_v(v)=v
$$
I worked out exactly the same math as wikipedia and I ended up with
$$
(d \exp_p)_v(v) = \frac{d}{dt}\left. \left(\gamma((t+1),p,v) \right) \right|_{t=0}
$$
The equation is based on the use of the curve $\alpha(t) = (t+1)v$ where $v \in T_p M$. Apparently the key in understanding how to fill the gap is to use somehow the parallel transport but I couldn't figure from the given answer actually.
The specific bit I can't figure is that apparently, from one of the comments, it might be the case that the result of $(d \exp_p)_v(v)$ is actually the parallel transport of $v$ along the geodesic passing through $\exp_p(v)$.
Can anyone clarify?
 A: It might be easier than you think if you stick to do Carmo's text. The key claim is that
$$
\langle d(\text{exp}_p)_v(v), d(\text{exp}_p)_v(w_T) \rangle = \langle v,w_T \rangle,
$$
for any $w_T = av \in T_pM$.  By the fact that $\gamma(t) = \text{exp}_p(tv)$, we can compute $d(\text{exp}_p)_v(v)$ by taking $v$ as the initial velocity of the curve $\alpha(t) = v+vt$ starting at $v$. We will obtain $d(\text{exp}_p)_v(v) = \gamma'(1)$. So we have
\begin{align}
\langle d(\text{exp}_p)_v(v), d(\text{exp}_p)_v(w_T) \rangle &= a \, \langle d(\text{exp}_p)_v(v), d(\text{exp}_p)_v(v) \rangle \\ 
&=a \, \langle \gamma'(1),\gamma'(1) \rangle \\ 
&= a\, \langle \gamma'(0),\gamma'(0) \rangle\\
&= a\,  \langle v,v \rangle\\
&= \langle v,w_T \rangle,
\end{align}
where the third equality holds because $\langle \gamma'(t),\gamma'(t) \rangle$ is constant along geodesic $\gamma$.
A: Here's an attempt to answer, or at least to prove part of Gauss's lemma without needing to claim that $d(\text{exp}_p)_v(v) = v$. This equation makes me a little worried because $v \in T_p M$, whereas $d(\text{exp}_p)_v(v) \in T_{\text{exp}_p(v)}M$ and so a priori equating them doesn't make sense. What I assume is happening is that they are abusing notation and writing this equation for the shorthand of "$d(\text{exp}_p)_v(v)$ is the parallel transport along $\text{exp}_p(tv)$ of $v$". Another answer talking more about this to correct me/shed some more light on this would be wonderful, especially if this isn't the case.
Like you wrote, we want to evaluate this derivative by writing it as the time derivative, evaluated at a certain time, of a path which at that time is $v$ and has derivative $v$. It's convenient to write it as follows:
$$
d(\text{exp}_p)_v(v) = \frac{d}{dt}|_{t = 1} \text{exp}_p(t v).
$$
Note that $t \mapsto tv$ is $v$ at $t = 1$ and has time derivative $v$ everywhere. But $\gamma : t \mapsto \text{exp}_p(tv)$ is the geodesic starting at $p$ which has initial velocity $v$. If we define $V(t)$ to be the velocity vector field along $\gamma$ then we know that $\nabla_{V}V = 0$. Then $d(\text{exp}_p)_v(v)$ is the velocity vector of $\gamma$ at $t = 1$, and we have
$$
\frac{d}{dt}g(V(t), V(t)) = 2g(\nabla_V V, V) = 0
$$
since $V$ is parallel transported along $\gamma$. This proves that $g(d(\text{exp}_p)_v(v), d(\text{exp}_p)_v(v)) = (v, v)$. I believe that this is all you need to prove Gauss's lemma, because I think you just decompose the claim $g(d(\text{exp}_p)_v(v), d(\text{exp}_p)_v(w))$ into the cases when $w = \lambda v$ (in which case we proved it above) and $w$ is orthogonal to $v$ (in which case I don't think (?) you need the assertion about the equality of $d(\text{exp}_p)_v(v)$ and $v$).
