Relevance / Importance of the category Mat I am new, very new, to category theory - which I am trying to learn on my own.  I've run across an example in several texts which confuses me.
Specifically, it is the category Mat where objects are natural numbers and morphs between the objects m and n are mxn matrices.  I can see that this is a category, but I wonder what it provides an example of.  While definitions are neither correct nor incorrect, they can certainly be useful or useless, and this example seems to be in the latter category.
If authors want an example of a category with matrices as morphs, why not let the objects also be matrices, subject only to the restriction that the dimensions are such that post multiplication of a matrix in the domain by a morph gives a matrix in the codomain?  This gives a much richer example.
Again, I am not questioning the "correctness" of anything.  But when I encounter what seems like a trivial example, used repeatedly, I have to wonder whether I am missing the whole point the author is trying to make.
Thanks for any help or comments.
 A: Mat is basically the same category as Vect of finite dimensional with a distinguished basis. It's to demonstrate the notion of categorical equivalence as well as a cute example of a category whose morphisms are not functions.
A: Is that category really so trivial? I think part of the point is that while the objects are rather "boring", there is much more richness to be found in the morphisms.
A: In my opinion, the two most significant things to learn from the example are:
Morphisms are important
One thing I really like about Mat is that it's a familiar example where our prior experience is to consider all of the importance to be in the morphisms, which serves to contrast with other examples like AbGrp where prior experience tends to be object-centric.
A running theme throughout category theory is the emphasis on the importance of morphisms — a point of view that can even be fruitfully taken to the extreme of considering objects to be wholly irrelevant beyond their role of being the sources and targets of morphisms.
E.g. in many of the familiar examples, all of the usual structure of the objects can be recovered from morphisms. For example, in Top, given a topological space $X$ one can identify its set of points with $\hom(1, X)$ and its family of open sets with $\hom(X, 2)$, where $1$ is the one-point space and $2$ is the Sierpinski space.
Categories are natural structures
Another thing I like about Mat is it demonstrates a new way in which categories naturally organize familiar structure.
Matrix algebra is sort of an oddball in abstract algebra because the product is only partially defined; it doesn't really fit well with the usual approaches to the subject; you have to do unnatural things like restrict your attention only to square matrices of fixed dimension, or do weird things like allow all products but define those with mismatch to multiply to zero.
But lo and behold, the structure of a category happens to be exactly how matrices with the matrix product want to be organized.

Another point about the example is it paves the way of applying ideas of category theory to linear algebra.
It wasn't until I learned about Mat, for example, that I really accepted that "column space of an $n \times m$ matrix" is a better notion for computation than "subspace of $\mathbb{R}^n$". Or finally allowed "$n \times 1$ matrix" to replace "element of $\mathbb{R}^n$" in my mind when doing calculations.
A: As already pointed out, $\mathbf{Mat}$ gives a fairly familiar example of a category with objects that aren't "sets equipped with" and with morphisms that aren't "functions such that". But so does yours. So what does make them different?
The very fundamental property of $\mathbf{Mat}$ (here I suppose $0$ is taken, and that entries of the matrices are in a field $k$) is that it is a skeleton of the category of finite-dimensional vector spaces over $k$ (as long as a strongly enough axiom of choice is assumed). A skeleton $\mathbf S$ of a category $\mathbf C$ is a full subcategory of $\mathbf C$ such that: every object of $\mathbf C$ is isomorphic to one of $\mathbf S$; and two isomorphic objects of $\mathbf S$ are in fact equal. It kind of reflects the way you think most of the time in linear algebra: when you say "Let fix basis for $V,W$ and ...", you basically say that you now work inside $\mathbf{Mat}$ rather than the category of finite-dimensional vectors spaces.
And this happens in other area of mathematics. Suppose you're studying the combinatorial class of finite labeled graphs: there is a functor $\mathbf{Bij}\to\mathbf{Set}$, where $\mathbf{Bij}$ has as objects the finite sets and as morphisms the bijections between such, mapping each finite set $S$ to the set of graphs whose vertices are labeled by elements of $S$. Then sentences like "Up to renaming the vertices by $1,\dots,n$" is a way to say that you are exploiting a skeleton of $\mathbf{Bij}$, which is the category $\mathbf P$ whose objects are natural numbers $n\geq 0$ and where $\mathbf P(n,n)$ is the permutation group on $n$ letters (and $\mathbf P(m,n) = \emptyset$ whenever $m\neq n$).
Now, I want to emphasise that your example is very different in nature: you define the category $\mathbf M$ whose objects are the matrices and where $\mathbf M(M,N) = \{P : PM = N\}$. It is far from skeletal, as every invertible $P$ gives a isomorphism from $M$ to $PM$ (and such are rarely equal). And this is clearly not equivalent to vector spaces. But in fact, this is still related to $\mathbf{Mat}$ somehow. For each category $\mathbf C$, you can define its category of arrows $\mathrm{Arr}(\mathbf C)$ whose objects are the morphisms of $\mathbf C$ and the morphisms between $f$ and $g$ are the commutative squares with $f$ on the top and $g$ on the bottom. So $\mathrm{Arr}(\mathbf{Mat})$ has matrices as objects and $\mathrm{Arr}(\mathbf{Mat})(M,N) = \{ (P,Q) : PM = NQ \}$. Hence your $\mathbf M$ is the subcategory of it containing all the objects but only the morphisms of the form $(P,\mathrm{Id})$. 
