If I understood your problem correctly, I would suggest the following approach:
First, go to this link and check the transformations that are given there for 2D.
Next try to write "decompose" your transformation matrix into those types of transformations because it is easier to see the inverse for them.
Since
Maybe two examples will help. As a first case (the easy one) take the matrix
$$\begin{bmatrix} 0 & 2 \\ 2 & 0\end{bmatrix}$$
It is a basic case, but I emphasizes the approach. I believe that you can see that this matrix can be written as
$$\begin{bmatrix} 2 & 0 \\ 0 & 2\end{bmatrix}\begin{bmatrix} 0 & 1 \\ 1 & 0\end{bmatrix} = \begin{bmatrix} 2 & 0 \\ 0 & 2\end{bmatrix}\begin{bmatrix} \cos(\pi/2) & \sin(\pi/2) \\ \sin(\pi/2) & \cos(\pi/2)\end{bmatrix} $$
which is a rotation by $\pi/2$ and a rescaling by a factor of $2$.
In the second example I will take the matrix
$$\begin{bmatrix} 2 & 1 \\ 0 & 2\end{bmatrix}$$
I see, using the wiki link that this looks like a shear parallel to the $x$ axis so maybe I should take out an identity matrix for the scaling as in the first example such that
$$\begin{bmatrix} 2 & 0 \\ 0 & 2\end{bmatrix}\begin{bmatrix} 1 & 0.5 \\ 0 & 1\end{bmatrix}$$
This indicates that I can use a shear and rescaling for that.
Now you have the transformation decomposed in more basic transforms and you should be able to see step by step what to do in order to go from the initial vector to the transformed one.
Note: It might be useful to take into consideration the existence of the statement $(AB)^{-1} = B^{-1}A^{-1}$.
EDIT:
I will take the general scenario of a matrix
$$A = \begin{bmatrix} a & b \\ c & d\end{bmatrix}$$
from which the following calculations can be made. I mention first that whenever rescaling occurs, I will just use the scaling value for ease, not the identity matrix multiplied by the value. Also I'll put below a list of all the transformations that I will use (and their inverse):
- scaling by $k$: $k \rightarrow k^{-1}$
- stretching by $k$: $\begin{bmatrix} k & 0 \\ 0 & 1\end{bmatrix} \rightarrow \begin{bmatrix} 1/k & 0 \\ 0 & 1\end{bmatrix}$ or $\begin{bmatrix} 1 & 0 \\ 0 & k\end{bmatrix} \rightarrow \begin{bmatrix} 1 & 0 \\ 0 & 1/k\end{bmatrix}$
- shearing by $k$: $\begin{bmatrix} 1 & k \\ 0 & 1\end{bmatrix} \rightarrow \begin{bmatrix} 1 & -k \\ 0 & 1\end{bmatrix}$ or $\begin{bmatrix} 1 & 0 \\ k & 1\end{bmatrix} \rightarrow \begin{bmatrix} 1 & 0 \\ -k & 1\end{bmatrix}$
Going back to the general matrix, first I factored $d$ such that
$$A = \begin{bmatrix} a/d & b/d \\ c/d & 1\end{bmatrix}d$$
Next I try to see what happens if I multiply to the matrix the inverse of a stretching matrix by $a/d$. This gives
$$A = \begin{bmatrix} 1 & b/d \\ c/a & 1\end{bmatrix}\begin{bmatrix} a/d & 0 \\ 0 & 1\end{bmatrix}d$$
Apply the inverse of a shear by $c/a$ on the first matrix from the above formula which implies that the following form is true
$$A = \begin{bmatrix} 1-(bc)/(ad) & b/d \\ 0 & 1\end{bmatrix}\begin{bmatrix}1 & 0 \\ c/a & 1\end{bmatrix}\begin{bmatrix} a/d & 0 \\ 0 & 1\end{bmatrix}d$$
Rescale by $1-(bc)/(ad)$ as the approach above to get
$$A = \begin{bmatrix} 1 & b/d \\ 0 & 1\end{bmatrix}\begin{bmatrix} 1-(bc)/(ad) & 0 \\ 0 & 1\end{bmatrix}\begin{bmatrix}1 & 0 \\ c/a & 1\end{bmatrix}\begin{bmatrix} a/d & 0 \\ 0 & 1\end{bmatrix}d$$
So now we have, from right to left, rescale, stretch, shear, stretch, shear. There might be other ways to write the same matrix using those basic transformations. Notice that I did not try to use the rotation.
Apply this to your example and it should work.