# 2nd variation of a family of curves in a Riemannian manifold

Let $\gamma:[a,b]\times[-\varepsilon,\varepsilon]\to M$ be a family of curves in a Riemannian manifold, with the first parameter denoted $t$, and the second denoted $s$. Suppose that $\gamma(a,\cdot)$ and $\gamma(b,\cdot)$ are constant. Then, if we let $$E(s)=\frac12\int_a^b\lvert\dot\gamma(t,s)\rvert^2\,\mathrm{d}t$$ then I wish to calculate $\frac{\mathrm{d}^2}{\mathrm{d}s^2}E(0)$ in the specific case where $\gamma(\cdot,0)$ is geodesic. Now, it's not so hard to calculate that \begin{align} \frac{\mathrm dE}{\mathrm ds} &= \int_a^b\langle D_s\dot\gamma,\dot\gamma\rangle\,\mathrm dt = \int_a^b\langle D_t\gamma',\dot\gamma\rangle\,\mathrm dt \\ &= \int_a^b\frac{\mathrm d}{\mathrm dt}\langle\gamma',\dot\gamma\rangle\,\mathrm dt - \int_a^b\langle\gamma',D_t\dot\gamma\rangle\,\mathrm dt \\ &=- \int_a^b\langle\gamma',D_t\dot\gamma\rangle\,\mathrm dt \end{align} which allows us to calculate that \begin{align} \frac{\mathrm d^2E}{\mathrm ds^2}(0) &= -\int_a^b\langle D_s\gamma',D_t\dot\gamma\rangle\,\mathrm dt - \int_a^b\langle\gamma',D_sD_t\dot\gamma\rangle\,\mathrm dt \\ &= -\int_a^b\langle R(\dot\gamma,\gamma')\dot\gamma,\gamma'\rangle\,\mathrm dt + \int_a^b\langle\gamma',D_tD_s\dot\gamma\rangle\,\mathrm dt \\ &= -\int_a^b\langle R(\dot\gamma,\gamma')\dot\gamma,\gamma'\rangle\,\mathrm dt + \int_a^b\langle\gamma',D_tD_t\gamma'\rangle\,\mathrm dt \\ &= -\int_a^b\langle R(\dot\gamma,\gamma')\dot\gamma,\gamma'\rangle\,\mathrm dt + \langle\gamma',D_t\gamma'\rangle\vert_{t=a}^b - \int_a^b\lvert D_t\gamma'\rvert^2\,\mathrm dt \\ &= -\int_a^b\langle R(\dot\gamma,\gamma')\dot\gamma,\gamma'\rangle\,\mathrm dt - \int_a^b\lvert D_t\gamma'\rvert^2\,\mathrm dt \end{align} However, what I'm supposed to get is that \begin{align} \frac{\mathrm d^2 E}{\mathrm ds^2}(0) = \int_a^b\lvert D_t\gamma'\rvert^2\,\mathrm dt - \int_a^b\langle R(\dot\gamma,\gamma')\gamma',\dot\gamma\rangle\,\mathrm dt + \langle D_s\gamma',\dot\gamma\rangle\vert_{t=a}^b \end{align} but this is pretty different from what I got. How did I go wrong?

First, you should say what you mean by $\dot\gamma$ and $\gamma'$. From your calculations, I assume you're using the convention that $$\dot\gamma(t,s) = \frac{\partial}{\partial t}\gamma(t,s), \qquad \gamma'(t,s) = \frac{\partial}{\partial s}\gamma(t,s).$$
Next, because you've stipulated that $\gamma(a,s)$ and $\gamma(b,s)$ are constant (i.e., independent of $s$), it follows that $\gamma'(a,s)\equiv 0$ and $\gamma'(b,s) \equiv 0$, and therefore $D_s\gamma'(a,0) = 0$ and $D_s\gamma'(b,0)=0$. So the boundary term in the formula you're aiming for is actually zero.
With this understanding, your formula is exactly the negative of the correct formula, once you take into account the symmetry of the Riemann curvature tensor: $$\langle R(\dot\gamma,\gamma')\dot\gamma,\gamma'\rangle = - \langle R(\dot\gamma,\gamma')\gamma',\dot\gamma\rangle .$$
Your sign mistake is in the second line of your computation of the second variation. The formula for commutation of second covariant derivatives is $$D_s D_t \dot\gamma - D_t D_s \dot\gamma = \pm R(\dot\gamma,\gamma')\dot\gamma,$$ with the sign on the right-hand side depending on which definition of the curvature tensor you're using.