The motivation for a tangent space I am trying to come to grips with the motivation for a tangent space, and was wondering if anybody could offer any help. Suppose we have a manifold $M$ and a function $f:M \rightarrow \mathbb{R}$. Now, just for a simple example, suppose that $M$ is the unit circle in $\mathbb{R}^{2}$, which we know is a $1$-dimensional manifold. Consider two points on the unit circle, $x$ and $y$.
Now, if I were to ask, what is the directional derivative of $f$ at $x$ in the direction of $y$,then because $f$ is technically defined on (a subset of) $\mathbb{R}^{2}$, I can take this directional derivative in the freshman calculus way by considering the straight arrow from $x$ to $y$ and calculating the directional derivative from that. 
However, if we consider $x$ and $y$ as points along this circle, then the direction from $x$ to $y$ is NOT a straight arrow in the $\mathbb{R}^{2}$ plane, but rather looks like a "curved arrow" of sorts because we must travel along the circle. Therefore, I would need to find a way to take the directional derivative in the direction of this "curved arrow".
Is this the basic motivation for all of this tangent space stuff? I'm understanding the math/proofs, but it is the "why" that is escaping me.
Thanks!
 A: Think of 'directions' on a manifold as local entities. Given a point $x$ on a manifold $M$ there is a small region $U\subseteq M$ about the point where each point in $U$ can be ascribed coordinates so that every point in $U$ gets a unique coordinate. For an $n$-dimensional manifold each point will get $n$ coordinates. Additionally, if you move smoothly around $U$ then the coordinates change smoothly. 
The utility of this is that you can think of $U$ as just being like $\Bbb R^n$ so you can directionally differentiate functions $U\to\Bbb R$ as though they were functions $\Bbb R^n \to \Bbb R$ by differentiating with respect to the ith coordinate. Since differentiation only relies on the behaviour of a function around a point, there are no issues caused by the fact that $M$ only resembles $\Bbb R^n$ around a small region of $x$.
The big difference in the case of manifolds is that the way you ascribe coordinates is arbitrary (up to diffeomorphism) so the meaning of directional differentiation then depends on the way you have assigned coordinates. However, since you are describing the behaviour as a function as you 'move' along the manifold, it should be possible to describe directional differentiation without relying on coordinates. After a bit of technical work, you can rest assured that the actual operation of directionally differentiating is intrinsic to the manifold itself. Then we call that operator a tangent vector. The space of all such operators is called the tangent space to $M$ at $x$.
Also note that your example doesn't really help you as the fact the circle is 1 dimensional means that there is only one way you can move on the manifold locally (i.e in coordinates only 1 coordinate can change). As a result, the tangent space to a circle at a point is 1 dimensional. 
A: Your question is very wide and philosophical! I give it a try.
There are several equivalent définition of the Tangent space at a point $x\in 
M$ where $M$ is a differential manifold. Recall that the notion of differentiable function is given chart-wise and do not use the tangent space.
First definition see for instance "semi-riemannian geometry" of Barett O'neil :
Let $M$ be a differential manifold and $x\in M$. 
Let $\mathcal C^1(M,\mathbb R)$ the set of continuously differntiable function from $M$ to $\mathbb R$. 
The Tangent space $T_xM$ of $M$ at $x$ is 
the set of $\mathbb R$-linear function $u : \mathcal C^1(M,\mathbb R)\rightarrow \mathbb R$ such that 
$$ \forall f,g\in \mathbb R, \quad u(fg)=f(x)u(g)+u(f)g(x) $$
Second definition : 
Let $M$ be a differential manifold and $x\in M$. Let 
$$E_x:=\{\gamma\in\mathcal C^1(]-1,1[,M) ~|~ \gamma(0)=x \}$$ the set of continuousy differentiable curve in $M$. Define the equivalence relation $ \gamma_1\sim \gamma_2 $ 
if their exists a chart $\phi : \mathcal U \rightarrow \mathcal V\subset \mathbb R^n$ around $x$ such that $(\phi \circ\gamma_1-\phi\circ \gamma_2) : ]-1,1[\rightarrow \mathbb R^n$ has derivative zero at $0$.
To come back to your question, the straight line between $x$ and $y$ is not well defined in your situation. You would need a riemannian metric or some additionnal geometric structure on you manifold. Another way to put it : $x$ does not see $y$ so their is no most direct way of going from the one to the other. However the second definition allows you to obtain the tangent space in a "path-wise way".
Now, from your argumentation (and the example you chose), I guess you are particularly acquainted to sub-manifold of $\mathbb R^n$. You can prove that every compact differentiable manifold is a submanifold of $\mathbb R^N$ for $N$ big enough, so you can be tempted to consider sub-manifold instead of the abstract notion of manifold. However, the fact that a manifold can be embedded into $\mathbb R^n$ is almost never useful to answer interesting questions. You might as well ask wether we consider groups instead of subgroup of permutation groups since the former can allways be realized as the latter. There are many such example in mathematics (vector spaces versus $\mathbb R^X$, rings versus quotient rings of polynomial rings with coefficients in $\mathbb Z$,...). 
Mathematicians are linguist that construct new languages to encapsulate their intuition of the problems they solve. 
There are plenty of operation that are not well suited for submanifolds :


*

*Quotient manifolds (they may be embedded in the same $\mathbb R^n$ as the manifold you first considered. You can embed $\mathbb R$ into $\mathbb R$ but the circle $\mathbb S^1 = \mathbb R/\mathbb Z$ can only be embedded into $\mathbb R^2$).

*You might want to consider several riemannian metric on a given manifold which make uneasy a common representation (think about Ricci flow and Poincaré conjecture). For instance, the circle you represented is the "round circle" but ellipse are also "differential circles" and from the manifold view point, they cannot be distinguished. The picture has lot of additionnal properties you may not want and might not be relevant to your problem

*Fiber bundles. You may not know that (yet) but these are very very important objects. Many geometrical structures are section of fiber bundles, they are omnipresent in High Energy physics or General Relativity. Classification of manifolds make systematic use of fiber bundles (Chern's proof of Gauss-Bonnet-Chern is a basic example). Many operation you can do on Vector spaces can be done on Vector fiber bundle over a given manifold (direct sum, product, tensor product, quotients,...). 

*One broad class of example of manifolds is given by Lie groups and their quotients. The theory makes uses of quotients, covering, group sum,...

A: There are various technically different approaches to introduce the tangent space $T_xM$ of a manifold $M$ at a point $x$.
For motivational purposes, in my opinion the best starting point is to consider a differentiable curve $u : J \to \mathbb{R}^n$, where $J \subset \mathbb{R}$ is an open interval. Then the tangent $\tau_{u,s}$ of $u$ at $s \in J$ is the line through $u(s)$ in direction $u'(s)$ which is parametrized by
$$\tau_{u,s}(\xi) = u(s) + \xi u'(s).$$
Here $u'(s) \in \mathbb{R}^n$ is the ordinary derivative of $u$. We call $u'(s)$ the tangent vector of $u$ at $s$.
Now let us consider a differentiable submanifold $M \subset \mathbb{R}^n$, for example $S^1 \subset \mathbb{R}^2$. What is a tangent vector of $M$ at $x$? It is any tangent vector $u'(s)$ where $u : J \to  \mathbb{R}^n$ is a curve in $M$ such that $u(s) = x$ (a curve in $M$ is one satisfying $u (J) \subset M$). It turns out that the set of all tangent vectors of $M$ at $x$ is a linear subspace of $\mathbb{R}^n$ whose dimension equals the dimension  of $M$. Let us denote this linear subspace by
$$\tilde{T}_x M .$$
For $M = S^1 \subset \mathbb{R}^2$  we obtain $\tilde{T}_x S^1 = \{ y \in \mathbb{R}^2 \mid y \cdot x = 0 \}$.
This construction only works for differentiable submanifolds $M \subset \mathbb{R}^n$. However, a simple observation allows us to get rid of the embedding $M \subset \mathbb{R}^n$.
Let $\mathcal{C}(M,x)$ denote the set of all differentiable curves $u : J \to M$ such that $0 \in J$ and $u(0) = x$. Let $i : M \to \mathbb{R}^n$ be the inclusion map. It is well-known that a function $u : J \to M$ is differentiable in the manifold sense (using charts on $M$) iff $iu : J \to \mathbb{R}^n$ is differentiable in the usual sense of multivariable analysis. We therefore get a surjective
function
$$d : \mathcal{C}(M,x) \to \tilde{T}_x M, d(u) = (iu)'(0) .$$
An equivalence relation $\sim$ on  $\mathcal{C}(M,x)$ is defined by $u_1 \sim u_2$ iff $d(u_1) = d(u_2)$. Writing
$$T_x M  =  \mathcal{C}(M,x)/\sim$$
we see that $d$ induces a bijection
$$\hat{d} : T_x M \to \tilde{T}_x M .$$
In other words, each tangent vector $v \in \tilde{T}_x M$ can be identified with an equivalence class of curves $u \in \mathcal{C}(M,x)$. Our definition of $\sim$ was based on the embedding $M \subset \mathbb{R}^n$. However, it is an easy exercise to show that $u_1 \sim u_2$ iff
$(\ast)$ $(\phi u_1)'(0) = (\phi u_2)'(0)$ for all charts $\phi : U \to \mathbb{R}^m$ on $M$ around $x$.
Here $\phi u_k$ is defined on $\phi^{-1}(U) \subset J_k$.
That is, we can define $\sim$ via $(\ast)$, and this works for an arbitrary $M$ so that $T_x M$ is always well-defined. This is the "correct" version of the tangent space of $M$ at $x$.
