Objects are finite sets, arrows are matrices. How is this a category? I just started to read this book on category theory. 
How is this example below a category? 
I have difficulty imagining what this construct really is.  
Could someone please illuminate me ?
I have a physics background, I am not a mathematician.
Perhaps with some very simple example or analogy that is understandable for a physicist's mind.
Thanks for reading.
EDIT:
Does this example make sense at all ?
EDIT 2:
What confuses me is that for a given object $A$, $i=|A|$ is fixed, so is $j=|B|$, 
so the location of a natural number in the matrix $F$ is determined by the size of $A$ and $B$. So the arrow from $A$ to $B$ is a number in a matrix and not the matrix itself !
EDIT 3: Many thanks for the answers ! I think I get it now.
Example:

As a reminder a category is defined as:

EDIT 4:
I just bought the latest eBook version where this mistake has been corrected:

 A: I think that this is indeed worded incorrectly. I assume the author really wanted to write $F=(n_{i\,j})_{1\le i\le\lvert A\rvert,1\le j\le\lvert B\rvert}$.
A: I think the intended example is this:
The objects are finite sets and given two objects $A$ and $B$, $hom(A,B)$ is the set of all $|A| \times |B|$ matrices with entries in $\mathbb{N}$ (you could replace $\mathbb{N}$ here by $\mathbb{Z}$, $\mathbb{R}$ or anything for which matrix multiplication is well defined and associative). Then it is straightforward to check the axioms. 
A: The category the author was intending to define is as follows.


*

*Objects. Finite sets

*Arrows. An arrow $f : A \rightarrow B$ is just a function $f : A \times B \rightarrow \mathbb{N}$.

*Composition. The composition of arrows $f : A \rightarrow B$ and $g : B \rightarrow C$ is the unique function $h : A \times C \rightarrow \mathbb{N}$ given as follows. $$h(a,c) = \sum_{b \in B}f(a,b)g(b,c)$$

*Identities. If $A$ is an object, then $\mathrm{id}_A$ is the unique function $A \times A \rightarrow \mathbb{N}$ given as follows.
$$\begin{align} a=a' & \;\rightarrow\; \mathrm{id}_A(a,a')=1 \\ a\neq a' & \;\rightarrow\; \mathrm{id}_A(a,a')=0 \end{align}$$
To see what this has to do with matrices, just imagine that every object of this category is not a set, but rather a totally ordered set. Then an arrow $f : A \rightarrow B$ in this category, which as you'll recall is just a function $f : A \times B \rightarrow \mathbb{N},$ can be visualized as an $|A| \times |B|$ array of natural numbers.
A: I think that this is what is going on:
The objects are finite sets.
For the morphisms: given two sets $A$ and $B$ you want a set $Mor(A,B)$ of elements/arrows $f: A \to B$. So here $A$ is the domain and $B$ is the codomain of $f$. This needs to satisfy the composition law.
Here you have for two finite sets $A$ and $B$ this set of morphisms consists of all $\lvert A \rvert\times\lvert B\rvert$ matrices (with entries in $\mathbb{N}$). That is, a morphism/arrow is exactly a matrix. You compose two morphisms/arrows by multiplying the matrices. So The composition of morphisms/arrows is given by matrix multiplication so that if $f \in Mor (A,B)$ and $g\in Mor(B,C)$, then $g\circ f\in Mor(A, C)$. Is this well defined? Yes, because $Mor(A,C)$ consists exactly of $\lvert A \rvert\times\lvert C\rvert$ matrices and you get that from multiplying $\lvert A \rvert\times\lvert B\rvert$ matrices with $\lvert B \rvert\times\lvert C\rvert$ matrices.
