I understand your frustration with wanting to "visualize" matrices (even simple $2 \times 2$s)!
So, let's start first with the complex $1 \times 1$ matrices $[z]$ with $z \in \mathbb{C}$. Let's also constrain this set of matrices by imposing a rule that each of these matrices must be unitary; namely, $z$'s conjugate transpose is also its inverse. So $\bar{z} = z^{-1}$ implies $|z| = 1$, which means the set of all such $1 \times 1$ matrices is in fact the 1-dimensional unit circle. If we replace $\mathbb{C}$ with $\mathbb{R}$, we get two points: $\lbrace -1, 1 \rbrace$ (the 0-dimensional "circle"). If we lose the unitary requirement, then the real $1 \times 1$ matrices just form an infinite line: $\mathbb{R}$.
Ok, now let's move up to $2 \times 2$ real matrices. We know such matrices belong in $\mathbb{R}^{4}$ (in fact, all together they are $\mathbb{R}^4$), but how do we make sense of them?
Let's consider all such matrices (real and $2 \times 2$) which are also invertible. They form an open subspace of $\mathbb{R}^4$ that is disconnected: one component represents the matrices of positive determinant; the other negative. Ok, so this isn't very helpful, but if you think of these matrices as what they really are (linear transformations), then it will begin to make sense.
Note that a linear transformation does two things to a vector: (1) it scales it, and (2) it rotates it. Then we get a new vector. So we are sending $\mathbb{R}^2 \longrightarrow \mathbb{R}^2$. Scaling can be parameterized by a single real number $\lambda$; similarly, rotation can be parameterized by a single real angle $\phi \in \mathbb{R} / \pi\mathbb{Z}$ (in fact, the set of all real $2 \times 2$ rotation matrices is the unit circle). These two parameters can be thought of as your extra $2$ dimensions; namely, to visualize an invertible real $2 \times 2$ matrix $A$, you could view it as a $4$-tuple $(x, y, \lambda, \phi)$ ($\in \mathbb{R}^4$), where $v = [x, y]^\top$ is a vector in $\mathbb{R}^2$, and $\lambda$ and $\phi$ are the scaling and rotating components that will send $v$ to $Av$.
Another kind of hand-wavy way is to say that a $2 \times 2$ real invertible matrix can be decomposed into it's scaling part and it's rotating part (each independent). The rotating part, as claimed above, is really just the circle $S^1$. The scaling part is really just the real line $\mathbb{R}$. So together they form sort of an infinite cylinder $\mathbb{R} \times S^1$. So the "total space" is $\mathbb{R}^2 \times \mathbb{R} \times S^1 = \mathbb{R}^3 \times \mathbb{R}/\mathbb{Z}$ which is "in" (hand-wavy) $\mathbb{R}^4$.
In line with the door comment, this is a space in which the vector $v$ under transformation by $A$ is "preserved" through all states until it reaches its final state $Av$.