Why are the objects of $\mathbf{Mat}_\mathbb K$ the natural numbers? I came across this in the context of a functor $F:\mathbf{FdVect}_\mathbb K\rightarrow\mathbf{Mat}_\mathbb K$ where we map arrows (linear maps) to their respective matrices in a given basis and each object $\mathbb K ^n$ just to $n$. This all makes sense to me, except that as far as I can see matrices do not have type $m\rightarrow n$ where $n,m\in\mathbb N$. The text (Categories for the Practising Physicist) makes the following comment:
it strongly emphasizes that objects are but labels with no internal structure.
however if we merely need a set with the right cardinality wouldn't $\mathbb N$ do just as well as $\{\mathbb K ^n:n\in\mathbb N\}$ for $\mathbf{FdVect}_\mathbb K$?