Dimension of $m\times n$ matrices

I'm a bit confused on the notion of the dimension of a matrix, say $\mathbb{M}_{mn}$. I know how this applies to vector spaces but can't quite relate it to matrices.

For example take this matrix: $$\left[ \begin{array}{ccc} a_{11}&\cdots&a_{1n}\\ \vdots & \vdots & \vdots \\ a_{m1}&\cdots&a_{mn} \end{array} \right]$$

Isn't the set of matrices $\mathbb{M}_{mn}$ with exactly one entry $a_{ij}$ set to $1$ on each matrix and $m\times n$ total matrices a basis for $\mathbb{M}_{mn}$? In the sense that we can take some linear combination of them and add them up to create:

$$a_{11} \cdot \left[ \begin{array}{cccc} 1&\cdots&\cdots&0\\ \vdots & \vdots & \vdots & \vdots \\ 0&\cdots&\cdots&0 \end{array} \right] + a_{12} \cdot \left[ \begin{array}{cccc} 0 &1 &\cdots&0\\ \vdots & \vdots &\vdots & \vdots \\ 0&0&\cdots&0 \end{array} \right] + \cdots = \left[ \begin{array}{ccc} a_{11}&\cdots&a_{1n}\\ \vdots & \vdots & \vdots \\ a_{m1}&\cdots&a_{mn} \end{array} \right]$$

So the dimension of all $\mathbb{M}_{mn}$ is $m\times n$?

Yes, that is a basis for matrices, and yes that shows the dimension is $mn$.
If you still want to think in terms of column vectors, imagine taking each column of a matrix and stacking them all on top of each other to get a column vector with $mn$ entries.
The term ''dimension'' can be used for a matrix to indicate the number of rows and columns, and in this case we say that a $m\times n$ matrix has ''dimension'' $m\times n$.
But, if we think to the set of $m\times n$ matrices with entries in a field $K$ as a vector space over $K$, than the matrices with exacly one $1$ entry in different positions and all other entries null, form a basis as find in OP, and the vector space has dimension $m n$.