Can't we consider the curve $t\to(\gamma(t),X(\gamma(t))$ instead of the covariant derivative $\nabla_{\gamma(t)}X$? Many text books on differential geometry motivate covariant derivative more or less by saying that if you have a vector field along a curve on a manifold (that is a curve $\gamma(t)$ and an assignment of a vector $X(\gamma(t))$ at each point) then you can not directly define its derivative because you can not subtract two vectors living at different spaces.
Lie Derivative here does not also help since you would need to extend $\dot{\gamma(t)}$ to a vector field to define the Lie derivative along that vector field and then the Lie derivative will depend on the extension.
So ok covariant derivative $\nabla_{\gamma(t)}X$ gives you a way to differentiate vector fields along curves by letting you compare two different tangent spaces through parallel transport along $\gamma(t)$. But what I dont understand is what is the problem with constructing the curve
$t \rightarrow (\gamma(t),X(\gamma(t)))$ which will be a curve inside the manifold TM and then derivative it whose coordinate expression would be $(\gamma(t),X(\gamma(t)),X(\gamma(t)),\beta(t))$ and call $\beta(t)$ the derivative of $X$ along $\gamma(t)$. This derivative does not live on $TM$ but lives on $TTM$ that is true, but what is the problem with this?
This also makes me think whether if one can define a connection on $M$ by defining some kind of projection $\pi: TTM \rightarrow TM$ so that first you find $\beta$ as above and then somehow send it back to $TM$. In fact this is the way how you turn a second order ODE on $M$ to a first order ODE on $TM$. Is there are more deeper way of understanding the necessity for covariant derivative?
 A: Your suggestion is a sound one, and in fact the problem is not the identification of $TTM$ and $TM$, since it is locally trivial. That could be overcome, perhaps. The problem is that the lift you want is called a horizontal lift of your vector field, and that is precisely what you don't canonically have. A connection (which is the "mother" of the covariant derivative) does precisely that: it selects a horizontal space on each tangent space to the tangent bundle. The covariant derivative then works precisely the way you suggested, except it depends on that initial choice of connection.
For reference, see:
http://en.wikipedia.org/wiki/Ehresmann_connection
A: I was able to give a partial answer to my question, the part where I ask about the relation between linear connection on $M$ and Ehresmann connections on $TM$. It goes as follows. Given a linear connection on $M$, I will define a Ehresmann connection on $TM$ for which if a path $(\gamma(t),v(t))$ has tangent vectors in the horizontal bundle of the Ehr. connection then $v(t)$ will be parallel along $\gamma(t)$. I will start by defining the connection 1-form. Fix a nbd $U$ and let $\Gamma_{ij}^k(x)$ be the connection coefficients. Let $(x,v)$ be local coordinates on $TM$. Define the local $TM$ valued 1-forms on $TM$ by:
$$\eta|_{(x,v)} = dv^i \otimes \frac{\partial}{\partial x^i} + (\Gamma^k_{ij}(x)v^i)dx^j \otimes \frac{\partial}{\partial x^k}$$
One can show that this one form is globally defined if and only if $\Gamma^k_{ij}$ satisfies the coordinate transformation rules of the connection coefficients, which it does. Therefore it is a well defined 1-form on $TM$. Moreover in terms of the locally defined connection 1-form
$\omega(X) =(\Gamma^k_{ij}(x)X^i)dx^j \otimes \frac{\partial}{\partial x^k}$ this can be written as 
$$\eta|_{(x,v)} = dv^i \otimes \frac{\partial}{\partial x^i} + \omega(v)$$
hence the reason why these are called connection 1-forms. Now it is also to see that if $\beta(t)=(\gamma(t),v(t))$ is a curve such that $\dot{\beta}(t)$ belongs to the horizontal space defined by $\eta$ i.e $\eta(\dot{\beta}(t))=0$ then $v(t)$ is parallel along $\gamma(t)$. And geodesics are curves which are self parallel then. One can therefore go in the reverse direction to start with an Ehresmann connection on $TTM$ and induced a connection on $M$ which contains all the linear connection as well as what one could call "non-linear" connections. I am still trying to understand though what is the significance of trying to force the derivative of $v(t)$ to live in $TM$ rather than $TTM$. All this process also resembles the method of embedding the manifold in some $\mathbb{R}^n$, taking derivatives of $v(t)$ which possibly lie out of $TM$ and project them back to $TM$. I suspect it is the same kind of logic. I am still also puzzled by this process. Because using the differentiable structure on $TTM$ and $TM$ one can induce a connection on $M$ which is possibly derived from a metric. So why does one go through all the trouble of defining a metric and so on. I am also wondering if Ehresmann connection on $TM$ have relations with sub-Riemannian metrics on $M$.
