First, an explanation of the phenomenon. Then, we'll try for a rationale.
A quick review of how matrix multiplication: an $m \times n$ matrix with entries in $\Bbb F$ can be interpreted as a linear map from $\Bbb F^n$ to $\Bbb F^m$, where $\Bbb F^n$ denotes the set of column vectors of $n$ entries from $\Bbb F$. For example,
$$
A = \pmatrix{1 & 2 & 3\\4&0&2}
$$
is a $2 \times 3$ matrix. If we feed in any element of $v \in \Bbb F^3$, matrix multiplication produces an element of $\Bbb F^2$. That is, $A$ encodes the linear transformation $T_A:v \mapsto Av$. In our example,
$$
\pmatrix{1 & 2 & 3\\4&0&2} \pmatrix{1\\0\\-1} = \pmatrix{-2\\2}
$$
so, the linear transformation determined by $A$ takes the vector $v = (1,0,-1)$ to $w = (-2,2)$, which is to say $T_A(v) = w$.
Note, however, that all of these things are just boxes of numbers, including the column-vectors. In particular, $v = (1,0,-1)$ is also a $3 \times 1$ matrix, which means that it technically encodes a transformation from $\Bbb F^1$ to $\Bbb F^3$. So what gives? Does this linear map have anything to do with the column vector, or does this one symbol refer to two totally unrelated things?
The answer is obviously that first thing. In particular, the map encoded by $v$ takes elements of $\Bbb F^1 = \Bbb F$ to $\Bbb F^3$ by the map $T_v: \alpha \mapsto v\alpha$, which is to say that $T_v(\alpha) = \alpha v$. So for our example,
$$
T_v(2) = \pmatrix{1\\0\\-1}(2) = \pmatrix{2\\0\\-2}
$$
Clearly then, there is a natural correspondence between the linear maps from $\Bbb F$ to $\Bbb F^3$ and the vectors of $\Bbb F^3$. For every such linear map $T$, the vector $T(1)$ is an element of $\Bbb F^3$. Conversely, for every vector $v$, the map $T_v:\alpha \mapsto \alpha v$ is a linear map.
This association that we've made between $\Bbb F^{3 \times 1}$ and $\Bbb F^3$ is, in fact, an invertible linear map between vector spaces, which is to say it's an isomorphism of vector spaces. In fact, it's a natural isomorphism of vector spaces, which is to say that the same trick works with no changes no matter what your vector space is. If $V$ is a vector space, then $\mathcal L(\Bbb F,V)$ is "essentially" the same vector space.
In fact, in the context of matrices, this identification doesn't even change the notation we use for the computation! For example, let's take $A$ and $v$ as we did before, only this time, we're going to think of $v$ as really being the map $T_v:\Bbb F \to \Bbb F^3$. Now, matrix multiplication is not the application of a linear map, but the composition of two linear maps. In particular, the linear map $T_A \circ T_v:\Bbb F \to \Bbb F^2$ is define by
$$
(T_A \circ T_v)(\alpha) = T_A(T_v(\alpha)) = T_A(\alpha v) =\\
\pmatrix{1 & 2 &3\\4&0&2} \; \pmatrix{1\\0\\-1}\alpha =
\pmatrix{-2\\2}\alpha = \alpha w
$$
which is to say that $T_A \circ T_v = T_w$, and this follows directly from the fact that $T_A(v) = w$. Of course, since a column-vector already denotes a linear map, we could simply have expressed this computation by writing
$$
\pmatrix{1 & 2 & 3\\4&0&2} \pmatrix{1\\0\\-1} = \pmatrix{-2\\2}
$$
so on the surface, nothing has changed!
So, why do we care? In this context, it means that we can think of a vector space completely in terms of which linear maps go into that space. In other words, given a vector space $V$, knowing what $\mathcal L(\Bbb F, V)$ looks like is exactly the same as knowing the elements of $V$.
This trick of "knowing an object looks like without seeing its elements" turns out to be extraordinarily useful in applications of category theory. In particular, this trick comes up in logic, functional analysis, and abstract algebra.
In the language of category theory, we might say that the functor $\operatorname{Hom}_{\Bbb F}(\Bbb F, -)$ from the category of vector spaces to itself is an equivalence of categories.
