What is the precise statement of the theorem that connects matrices and linear transformations?

It's been awhile since I studied linear algebra, and I find myself not fully understanding the connection between linear transformations and matrices. So my question is:

What is the precise statement of the theorem that connects matrices and linear transformations?

In particular, what are the conditions for this theorem? Do we require that the dimension of the vector space be finite? What if its countable? And can the idea be generalized to the uncountable case?

-

Proposition. Let a field $K$ and two finite dimensional vector spaces $V,W$ over $K$ be given, and ordered $K$-bases $\def\B{\mathcal B}\B=(b_1,\ldots,b_m)$ of $V$ and $\def\E{\mathcal E}\E=(e_1,\ldots,e_n)$ of $W$. Define for any $K$-linear map $f:V\to W$ the $n\times m$ matrix $\def\Mat{\operatorname{Mat}}{}_\E\Mat_\B(f)=(a_{i,j})_{i=1,\ldots,n;~j=1,\ldots,m}$ with entries $a_{i,j}$ in $K$ by the requirement $$f(b_j)=a_{1,j}e_1+\cdots+a_{n,j}e_n \qquad \text{for j=1,\ldots,m}$$ (in other words column $j$ of ${}_\E\Mat_\B(f)$ contains the coordinates of $f(b_j)\in W$ in the basis $\E$). Then $f\mapsto {}_\E\Mat_\B(f)$ is a bijection $\mathcal L(V,W)\to M_{n,m}(K)$ and indeed an isomorphism of $K$-vector spaces.

Saying this is a bijection means that every matrix represents exactly one linear map, so representing linear maps by their matrix neither loses any information nor are there any matrices that represent no linear map at all. Saying it is an isomorphism of $K$-vector spaces means that taking sums and scalar multiples of linear maps translates into the same operations on their matrices.

Images by $f$ of general vectors of $V$ follow by linearity form the above equation: $$f(x_1b_1+\cdots+x_mb_m)=y_1e_1+\cdots+y_ne_n \qquad \text{where y_i=a_{i,1}x_1+\cdots+a_{i,m}x_m for i=1,\ldots,n},$$ the latter equations being represented more succinctly by the matrix-vector product $$\begin{pmatrix}y_1\\\vdots\\y_m\end{pmatrix} =\begin{pmatrix}a_{1.1}&a_{1,2}&\ldots&a_{1,m}\\ \vdots&\vdots&\ddots&\vdots\\a_{n,1}&a_{n,2}&\ldots&a_{n,m}\end{pmatrix}\cdot \begin{pmatrix}x_1\\x_2\\\vdots\\x_m\end{pmatrix}.$$ It follows that if $X$ is another finite dimensional $K$ vector space with basis $\def\F{\mathcal F}\F$, then for any $g\in \mathcal L(W,X)$ one has $${}_\F\Mat_\B(g\circ f)={}_\F\Mat_\E(g)\cdot{}_\E\Mat_\B(f),$$ in other words composition of linear maps corresponds to matrix multiplication.

Note the this final equation explains the otherwise somewhat curious placement of the basis subscripts to the $\Mat$ symbol that I have chosen.

As for the additional questions. Matrices are usually required to be of finite size (the possibility of actually writing them down being one of their merits), and that requires the vector spaces involved to be finite dimensional. The basic underlying fact however does not depend on finite dimensionality: any linear map $V\to W$ is competely determined by its images on a given basis of $W$, and any (properly indexed) family of vectors in $W$ occur as images of this basis for some linear map $V\to W$. So if you define matrices in $M_{\dim W,\dim V}(K)$ as having entries $a_{i,j}\in K$ for every pair $(i,j)$ of an index $i$ into a chosen basis of $W$ and an index $j$ of a chosen basis of $V$ such that for fixed $j$ only finitely many $a_{i,j}$ are nonzero (columns have finite support; this is necessary for them to be the coordinates of a vector), then one still gets an isomorphism between $\mathcal L(V,W)$ and $M_{\dim W,\dim V}(K)$, matrix multiplication is defined, and everything is fine as before. Countability is not an issue (but working with uncountable Hamel bases is usually a somewhat virtual activity). In algebra there is only one speed limit, $\aleph_0$.

-

This is only true for finite dimensional case. (As Sami said I rushed a bit, the problem is for infinite matrix the matrix multiplication is no always well defined, because you have infinite sums there).
And only if you have chosen a basis.

The idea is that a linear function is unique determined if you know it's values of basisvectors. Lets consider the case dimension is 3 and $e_1,e_2$ and $e_3$ are vectors of the basis. Than every vector $x$ can be written as $$x= \lambda_1 e_1 + \lambda_2 e_2 + \lambda_3 e_3$$ Let $L$ be our linear function, since it is linear we know that \begin{align} L(x)&=L(\lambda_1 e_1 + \lambda_2 e_2 + \lambda_3 e_3)\\ &=L(\lambda_1 e_1) + L(\lambda_2 e_2)+L(\lambda_3 e_3)\\ &= \lambda_1 L(e_1) +\lambda_2 L(e_2) + \lambda_3 L(e_3) \end{align} So when you know $L(e_1)$, $L(e_2)$ and $L(e_3)$ you know the values for all $x$. When you chose the basis $e_1$, $e_2,\ e_3$ you can write this as $$L(x) = \begin{pmatrix} L(e_1) & L(e_2) &L(e_3) \\ \end{pmatrix} \begin{pmatrix} \lambda_1\\ \lambda_2 \\ \lambda_3 \\ \end{pmatrix}.$$ Multiplying it out gives you the same as above. ($L(e_1)$ can be vectors too, that doesn't matter).
For infinite dimensional case matrices doesn't seem to make sense for me.

-
You rush to say that this is only true for finite dimensional case. See en.wikipedia.org/wiki/Matrix_(mathematics) – user63181 Mar 15 '13 at 8:03
@SamiBenRomdhane Thanks, but in infinite dimensional case the matrix multiplication is not always defined as the multplication yields infinite sums which could diverge – Dominic Michaelis Mar 15 '13 at 8:07
@DominicMichaelis I think it depends on the base ring. Maybe it's an odd example, but the product would be defined for infinite matrices over, e.g., the ring of formal power series with coefficients in a field. – A.P. Mar 15 '13 at 8:29
@A.P. yeah I think in several cases it will work fine, but I think at least the uncountable dimensional case will be pretty awkward as we need to guarantee that nearly every value is $0$ to have a chance at least of convergence. – Dominic Michaelis Mar 15 '13 at 8:37