With matrices, why does the Lie bracket agree with the matrix commutator? I gave an answer to Is there a group between $SO(2,\mathbb{R})$ and $SL(2,\mathbb{R})$? which was not popular.
Meanwhile, I found myself at a loss when wishing to explain why a matrix Lie group had, as a tangent space at $I,$ not only a vector space but a Lie algebra, with bracket agreeing with $[A,B] = AB-BA$ from matrix multiplication, that the matrix exponential of a bracket puts us back in the same group, and so on. I got what I think is a proof in one direction using this: http://en.wikipedia.org/wiki/Baker%E2%80%93Campbell%E2%80%93Hausdorff_formula#Matrix_Lie_group_illustration
http://en.wikipedia.org/wiki/Matrix_group
I do have Matrix Groups by Curtis, he does not go this far. I also have Foundations of Differentiable Manifolds and Lie Groups by Warner, which seems to do large portions of this.
Still, I am not entirely satisfied: with matrices, why does the Lie bracket agree with the matrix bracket, given a matrix group why is the identity tangent space both a vector space and a Lie algebra, given a tangent space at the identity that is a Lie algebra under matrix bracket, why do we get a group under matrix exponential?
 A: This quickly got longer than I thought it would be, and I'm sure I inadvertently skipped something. If you have any questions, feel free to ask.

Definition: (Lie Algebra) A Lie algebra is a real vector space $V$ equipped with an operation $$[\cdot,\cdot]:V\times V\to V,$$ called the bracket of the Lie algebra, such that

*

*$[\cdot,\cdot]$ is bilinear. That is, for $u,v,w\in V$ and $r\in\mathbb{R}$, $[u+v,w]=[u,w]+[v,w]$, $[u,v+w]=[u,v]+[u,w]$, and $[ru,v]=[u,rv]=r[u,v]$.

*$[\cdot,\cdot]$ is alternating. That is, for $v\in V$, $[v,v]=0$.

*$[\cdot,\cdot]$ satisfies the Jacobi identity. That is, for $u,v,w\in V$, $$[u,[v,w]]+[v,[w,u]]+[w,[u,v]]=0.$$

This is an abstract definition of a Lie algebra. As we can see, a Lie algebra is a vector space by definition.$^*$ But, what about the Lie algebra associated to a Lie group?
The product in the Lie algebra of a Lie group can be defined by using the typical Lie bracket $[X,Y]=XY-YX$ on vector fields, where we are treating vector fields as derivations mapping smooth functions to smooth functions. We simply extend $X_\mathrm{e}\in T_\mathrm{e}G$ to the vector field $X:g\mapsto X_g=(g)^L_*X_\mathrm{e}$, where $(g)^L_*$ is the pushforward of left multiplying by $g\in G$ (Acting on the left or right doesn't actually matter). Then, we can simply define $[X_\mathrm{e},Y_\mathrm{e}]=[X,Y]_\mathrm{e}$.

Proposition 1: The tangent space $T_\mathrm{e}G$ to the identity $\mathrm{e}$ of the Lie group $G$, equipped with the bracket operation defined in the previous paragraph, is a Lie algebra.

For the sake of helping you practice (Practice is the only way to truly understand Lie theory), I will leave this proof as an exercise. It is just a matter of checking the axioms. If you have trouble, I will be happy to help you below, in the comments.
There are several definitions of the exponential map. However, my primary choice when I go to prove something is the one I consider most useful. To give that definition, we need one more bit of terminology.

Definition: (1-parameter Subgroup) A 1-parameter subgroup on a Lie group $G$ is a Lie group homomorphism (a smooth homomorphism between Lie groups) between $\mathbb{R}$ and $G$.

Note, first and foremost, that this is NOT a subgroup. It is a homomorphism. Sometimes, it is useful to think of these as copies of $\mathbb{R}$ in $G$, and that is why the term comes up, but they are not themselves subgroups.
The set of all 1-parameter subgroups, on the other hand, does form a vector space. In fact, this vector space is isomorphic to $T_\mathrm{e}G$. The proof I know of this is rather tedious, but the intuition is pretty straightforward: there is a bijection between the tangent vectors (thinking as we did in high school of "magnitude and direction") in $T_\mathrm{e}G$ and the paths through $\mathrm{e}\in G$ with particular velocity ("magnitude and direction") at $\mathrm{e}$. With this intuition in hand, we define our exponential map.

Definition: (Exponential Map) Let $\mathfrak{g}$ be the Lie algebra of a Lie group $G$. Then, the exponential map $\exp:\mathfrak{g}\to G$ maps $X_\mathrm{e}\in\mathfrak{g}$ to $\theta(1)\in G$, where $\theta:\mathbb{R}\to G$ is the unique 1-parameter subgroup such that $\theta_*(\left.\frac{\partial}{\partial t}\right|_0)=X_\mathrm{e}$. (That is, $\theta$ is the unique 1-parameter subgroup of $G$ such that its tangent vector at $\mathrm{e}$ is $X_\mathrm{e}$.)

Thus, by definition, $\exp$ maps into $G$. But, what about the matrix exponential?

Proposition 2: The map $$\exp:\mathfrak{gl}_n(\mathbb{R})\to GL_n(\mathbb{R}),~X\mapsto\sum_{k=0}^\infty\frac{1}{k!}X^k$$ satisfies the definition of the exponential map.
Proof: Consider the map $\gamma_X:\mathbb{R}\to GL_n(\mathbb{R}),~t\mapsto\exp(tX)$, where $X\in\mathfrak{gl}_n(\mathbb{R})$ (so, $X$ can be any real $n\times n$ matrix). Note that $\gamma_X$ does, indeed, map into $GL_n(\mathbb{R})$, since $\exp(tX)^{-1}=\exp(-tX)$, so $\gamma_X(t)$ is invertible. Clearly, $\gamma_X$ is smooth, and $(\gamma_X)_*(\left.\frac{\partial}{\partial t}\right|_0)=X$. It remains to show that $\gamma_X(t+s)=\gamma_X(t)\gamma_X(s)$, to show that $\gamma_X$ is a homomorphism. But, a classic property of the matrix exponential is that $\exp((t+s)X)=\exp(tX)\exp(sX)$, so we are done. Thus, $\gamma_X$ is a 1-parameter subgroup, and $\gamma_X(1)=\exp(X)$ is the image of $X$ under the exponential map. Tombstone.

Thus, the matrix exponential is the exponential map of $GL_n(\mathbb{R})$ (and $GL_n(\mathbb{C})$, but I am sticking to reals for this explanation). Extending this, we get that the matrix exponential is the exponential map for all matrix Lie groups (matrix Lie groups are Lie subgroups of $GL_n(\mathbb{R})$ for some $n$).
Now, rather than prove to you that the matrix Lie algebra of each matrix Lie group is isomorphic to the abstract Lie algebra of that Lie group, I will show how we use the latter idea to pseudo-rigorously obtain the former, as I think this is more instructive. As before, if more rigor is requested, more will be given.
When I think about Lie algebras of Lie groups, I think of the elements of the Lie algebra as tangent vectors, which leads to the thought of "infinitesimal" group elements. For example, given the orthogonal group $O_n(\mathbb{R})$, we can think of its Lie algebra elements as imperceptibly small rotations. In other words, $X\in\mathfrak{o}_n(\mathbb{R})$ implies that, for some $\varepsilon > 0$ such that $\varepsilon^2=0$ (note that such an $\varepsilon$ doesn't exist), $(I+\varepsilon X)\,``\in"O_n(\mathbb{R})$, where $I$ is the identity matrix. But, since $A\in O_n(\mathbb{R})$ if and only if $AA^T=I$, $X\in\mathfrak{o}_n(\mathbb{R})$ "if and only if" $$(I+\varepsilon X)(I+\varepsilon X)^T=I+\varepsilon(X+X^T)+\varepsilon^2XX^T=I+\varepsilon(X+X^T)=I,$$ so $X+X^T=0$, which is the definition for $\mathfrak{o}_n(\mathbb{R})$. Thus, thinking of Lie algebra elements as tangent vectors gives us the matrix Lie algebra interpretation.
Since you have a reference-request tag, I will suggest J. Frank Adams's Lectures on Lie Groups, which provides an easily comprehendible introduction to abstract Lie groups. Also, Chapter 10 of Spivak's A Comprehensive Introduction to Differential Geometry, Volume 1 (I use the 3rd edition) is, as intended, fairly comprehensive.
$^*$For you fans of category theory, I mean that there is clearly a forgetful functor from each Lie algebra to its underlying vector space.
