Find the matrix of the transformation with respect to the given basis We have $\phi\colon \mathbb{R}^4 \to \mathbb{R}^4, \phi(x_1,x_2,x_3,x_4)= (x_1-2x_3,-2x_3-x_4,x_1+2x_3+2x_4,x_1+x_4)$. We want to find the matrix of this transformation with respect to the following basis of $\mathbb{R}^4$:
\begin{align}
    \alpha_1&=(1,1,0,0)\\ \alpha_2&=(0,1,1,0)\\ \alpha_3&=(0,0,1,1)\\ \alpha_4&=(0,0,0,1)\end{align}
First, we can write down the transformation as
$$
A_\phi = 
        \begin{bmatrix}
        1 & 0 & -2 & 0 \\
        0 & 0 & -2 & -1 \\
        1 & 0 & 2 & 2 \\
        1 & 0 & 0 & 1\\
        \end{bmatrix}
$$
and simplify it to the reduced row echelon form
$$
        \begin{bmatrix}
        1 & 0 & 0 & 1 \\
        0 & 0 & 1 & {1 \over 2} \\
        0 & 0 & 0 & 0 \\
        0 & 0 & 0 & 0\\
        \end{bmatrix}
$$
From this, we immediately see $\dim\ker\phi = 2$ and, by extension, $$\dim\operatorname{im} \phi = \dim\mathbb{R}^4 - \dim\ker \phi = 4 - 2 = 2.$$
So, we can choose whichever two linearly independent column vectors from $A_\phi$ and they will be a basis of $\operatorname{im}\phi$.
Now, how do we proceed from here to find the matrix of the transformation with respect to the given basis?
 A: You don’t proceed from here. Finding the rank and nullity of a matrix doesn’t really tell you anything that you’d need to know to perform a change of basis on it.  
Recall that the columns of a transformation matrix are the images of the domain basis vectors expressed relative to the basis of the codomain. So, for each basis vector $\alpha_j$, find the expression of $\phi\alpha_j$ as a linear combination $\sum_{i=1}^4 c_{ij}\alpha_i$. The required transformation matrix is then the coefficient matrix $[c_{ij}]$.  
This computation can be accomplished all at once via matrix multiplication. Let $B$ be the matrix with the vectors $\alpha_k$ (expressed relative to the standard basis) as its columns. The columns of $A_\phi B$ are then the images of these vectors, also expressed relative to the standard basis. Observe that $B$ can be interpreted as converting from the $\{\alpha_k\}$ basis to the standard one (why?), so $B^{-1}$ converts from the standard basis to the $\{\alpha_k\}$ basis. Thus, $B^{-1}A_\phi B$ is the required matrix. This operation is known as a similarity transformation or conjugation of $A_\phi$.
