Covariant derivative geometric interpretation I'm having some trouble understanding what the covariant derivative means geometrically. I know the definition which states that for a tensor T with any number of indices: $ \nabla_j T = \frac{\partial T}{\partial Z^k} + $ the Christoffel symbols contracted with each index of the tensor appropriately. 
In the case of an invariant, I can visualize this as the normal partial derivative, so the geometric interpretation is clear. Also, for a first-order tensor $V^i$ such that vector $ \textbf{R} = V^i \textbf{Z}_i$, I know that I can say $$ \frac{\partial \textbf{R}}{\partial Z^k} = \nabla_k (V^i) \textbf{Z}_i $$ However, how can I understand the covariant derivative of higher order tensors and of vectors? What is the geometric meaning? 
 A: @mollyerin gives a nice answer. Furthermore, depending on the rigor that you are looking for I think that Roger Penrose's book Road to Reality gives a nice overview of the geometric interpretation of covariant derivatives. It should be noted that the book attempts to present topics such as the covariant derivative and other mathematical subjects using layman's terms. 
A: I'll say a few words about how I think about covariant derivatives, which is really just expanding on janmarqz's comment (hopefully others will contribute their own viewpoints as well):
For me, the most important geometric idea behind a covariant derivative $\nabla$ is that given a curve $\gamma$ in a manifold $M$, $\nabla$ gives you an isomorphism between the tangent spaces $T_{\gamma(t_1)}M$ and $T_{\gamma(t_2)}M$ for any two points on the curve. Mathematically, this isomorphism
$$
   P : T_{\gamma(t_1)}M \to T_{\gamma(t_2)}M
$$
is the unique isomorphism with the property that for any $v \in T_{\gamma(t_1)}M$, there exists a vector field (which I'll call $v(t)$) along $\gamma$ such that $v(t_1) = v, v(t_2) = P(v)$, and $\nabla_{\gamma'(t)} v(t) = 0$ for all $t \in [t_1, t_2]$.
This isomorphism is called "parallel transport"; I like to picture a surface embedded in $\mathbb{R}^3$, such as the 2-sphere, and think of parallel transport along a curve $\gamma$ as "dragging" vectors along that curve. (Important remark: the isomorphism obtained depends on the choice of curve $\gamma$ in general.)
Of course, once you have an isomorphism of vector spaces, you get an isomorphism of any of the associated tensor spaces as well. So if $T$ is a $(k,l)$-tensor on $T_{\gamma(t_1)}M$, then we get a $(k,l)$-tensor $PT$ on $T_{\gamma(t_2)}M$.
Now the point is that once you have this "parallel transport" isomorphism, the covariant derivative $\nabla_X \mathcal{T}$ is a literal derivative in the following precise sense: Given a vector $X \in T_pM$, let $\gamma$ be any curve with $\gamma'(0) = X$, and let $P_t$ be the "parallel transport along $\gamma$" isomorphism
$$
  P_t : T_{\gamma(t)}M \to T_{\gamma(0)}M \quad (= T_pM).
$$
Then for any tensor field $\mathcal{T}$ on $M$,
$$
  \nabla_X \mathcal{T} = \frac{d}{dt}\Big|_{t=0} \Big( P_t \big( \mathcal{T}(\gamma(t)) \big) \Big). 
$$
This is a very precise interpretation of the idea that $\nabla_X \mathcal{T}$ gives you the derivative of $\mathcal{T}$ in the direction of $X$.
A: Let me add that in order to find curves of minimal length between to positions in a surface for example, one sets an equation like 
$$\nabla_{C'}C'=0,$$
which leads you to the classical expression 
$\ddot{u}^k+\Gamma^k{}_{st}\dot{u}^s\dot{u}^t=0$, where $C'=\dot{u}^s\partial_s$ is the tangent field for $C$. 
This is pretty geometrical.
