Is it coincidental that matrices are useful for solving systems of equations or is there a deeper (geometric) connection? Let's say we have this simple system of equations:
$$-x + y = 1$$
$$ x + y = 3$$
They intersect at the point $(1,2)$ and this is the graph:

One way of going about and solving this system of equations is to convert it into matrix form:
$$\begin{bmatrix}
-1 & 1\\
1 & 1
\end{bmatrix} * \begin{bmatrix}
x\\
y
\end{bmatrix} = \begin{bmatrix}
1\\
3
\end{bmatrix}$$
When my brain sees this matrix form, it has the following geometric meaning: If we apply the linear transformation encoded by $\begin{bmatrix}
-1 & 1\\
1 & 1
\end{bmatrix}$ to vector $\begin{bmatrix}
x\\
y
\end{bmatrix}$, then the output vector is $\begin{bmatrix}
1\\
3
\end{bmatrix}$.
We can then use Gaussian Elimination to determine $\begin{bmatrix}
x\\
y
\end{bmatrix}$.
My question: Is the matrix form of the system of equations simply a useful trick to algorithmically solve these systems of equations use Elementary Row Operations, etc. Or is there a deeper geometric connection between the intersection of two lines and the linear transformation of the vector that describes said intersection.
 A: You are right: matrices can be thought of as representing linear transformations. An $(m \times n)$-matrix takes as inputs vectors in an $n$-dimensional vector space and outputs vectors in an $m$-dimensional vector space. The crucial observation here is that in order to completely determine a linear transformation $T: V \to V'$, it suffices to know what it does on a certain chosen basis $\{e_1, \dots, e_n\}$ of $V$, since for any other $v \in V$, writing $v = \sum_{i=1}^n v_i e_i$ means that we have to get $T(v) = \sum_{i=1}^n v_i T(e_i)$. Still the $T(e_i)$ are vectors, but we would like to express these using numbers as well. So we chooose another basis $\{e_1', \dots, e_m'\}$ of $V'$, so that we can write any vector $v' \in V'$ as a linear combination of the $e_i'$. In particular, there are certain numbers $T_{ij}$ such that we have $T(e_i) = \sum_{j = 1}^m T_ij e_j'$. We conveniently put all these $n \cdot m$ numbers in a rectangular shape because it makes calculations easier, and tadá: we get our usual matrix notation.
A: Ok, I think I've got a potential explanation. The key is my realization that an equation can be thought of geometrically as a linear transformation of space.
Let's take our first equation: $$ -x + y = 1$$
This can be rewritten in "linear algebra" form:
$$\begin{bmatrix}
-1 & 1 
\end{bmatrix} *\begin{bmatrix}
x  \\
y  
\end{bmatrix} = 1$$
This form is saying that if we apply the
linear transformation encoded by $\begin{bmatrix}
-1 & 1
\end{bmatrix}$ to a 2D vector $\begin{bmatrix}
x\\
y
\end{bmatrix}$, then the 1D output "vector" is $\begin{bmatrix}
1
\end{bmatrix}$.
As it turns out, there are an infinite amount of 2D vectors that will satisfy the above expression. These vectors form the line described by our first equation.
Now when we add the second equation into our system, the question becomes: Is there a vector that satisfies BOTH linear transformations encoded by their respective equations? Indeed, the graphical intersection of these lines is that very vector.
Correct me if I'm wrong, but thinking of equations as linear transformations of space is a beautiful example of duality in mathematics.
