Linearity of push forward $F_*$ How can I prove the linearity of $F_*$? What does $F_*$ eat?

If $N$ is smooth manifolds and $F: M \to N$ is a smooth map, for each $p \in M$ we define a map $F_*: T_pM \to T_{F(p)}N$, called the push-forward associated with $F$, by
  $$(F_*X)(f) = X(f \circ F).$$

 A: It often times just helps to write out exactly what the formula you are trying to get in the end looks like, and then interpreting that.
For example, you are trying to show that $F_\ast(\alpha X)=\alpha F_\ast(X)$. So, what does the left-hand side do to elements of $T_{F(p)}N$? Well, as you pointed out
$F_\ast(\alpha X)(f)=(\alpha X)(f\circ F)$
what does the right-hand side look like?
$(\alpha F_\ast(X))(f)=\alpha F_\ast(X)(f)=\alpha X(f\circ F)$
Make sense?
EDIT(to answer Jellybean's questions in the comment): I am not entirely sure what you mean here. The tangent space of $M$ at $p$ is thought of, or at least the book you're reading thinks of it, as the $\mathbb{R}$-derivations of germs of smooth functions $\mathcal{G}_{M,p}$ at $p$. A smooth map $F:M\to N$ allows one to take a derivation $X$ of $\mathcal{G}_{M,p}$ and "push it forward" to a derivation of $\mathcal{G}_{N,F(p)}$. The way it does this is basically the observation that if you start with an element of $\mathcal{G}_{N,F(p)}$, germs of smooth functions on $N$ at $F(p)$, there is a way to pull this back to an element of $\mathcal{G}_{M,p}$. Namely, if $f\in\mathcal{G}_{N,F(p)}$ then $f\circ F\in\mathcal{G}_{M,p}$. Thus, if you start with a derivation $X$ of $\mathcal{G}_{M,p}$ you want to use this to produce a derivation of $\mathcal{G}_{N,F(p)}$. 
But, a derivation of $\mathcal{G}_{N,F(p)}$ eats in, well, obviously, elements of $\mathcal{G}_{N,F(p)}$. Thus, $X$ itself doesn't act on $\mathcal{G}_{N,F(p)}$. But, we observed above that we have a method of taking an element of $\mathcal{G}_{N,F(p)}$ and pulling it back to an element of $\mathcal{G}_{M,p}$, which is the set of things that $X$ does eat. So, to create a derivation of $\mathcal{G}_{N,F(p)}$, which let's call $F_\ast(X)$, of $\mathcal{G}_{N,F(p)}$ we merely take an element of $\mathcal{G}_{N,F(p)}$, pull it back to $\mathcal{G}_{M,p}$ by the rule $f\mapsto f\circ F$, and now that we're back on $X$'s diningroom table we feed it to $X$ to get a real number out. Thus, the derivation $F_\ast(X)$ all put together acts as $f\mapsto f\circ F\mapsto X(f\circ F)$. You can check that this definition of $F_\ast(X)$ does, in fact, produce a a derivation of $\mathcal{G}_{N,F(p)}$. 
Thus, the association $X\mapsto F_\ast(X)$ produces a map $F_\ast:T_p M\to T_{F(p)}N$. This map, the map $F_\ast$, is the pushforward. It is a linear map between tangent spaces. 
I hope that helps.
A: It follows from the definition of a directional derivative. If $F_*$ is linear then 
$$F_*(\lambda X + \mu Y) = \lambda (F_*X) + \mu (F_*Y)$$
If we apply the definition that $(F_*X)(\operatorname{f}) := X(\operatorname{f} \circ F)$ then we find that
$$[F_*(\lambda X + \mu Y)](\operatorname{f}) = (\lambda X + \mu Y)(\operatorname{f} \circ F)$$
Any definition of a connection involves linearity. In otherwords, the fact that 
$$(\lambda X + \mu Y)(\operatorname{f} \circ F) = \lambda [X(\operatorname{f}\circ F)] + \mu [Y(\operatorname{f} \circ F)] $$
is part of the definition of a connection.  Again, applying the definition, we have
$$\lambda [X(\operatorname{f}\circ F)] + \mu [Y(\operatorname{f} \circ F)] = \lambda (F_*X) + \mu (F_*Y)$$
Have a quick look at this article for the general definition of a connection.
