If matrices $A,B$ similar, find nonsingular $S$ s.t.$B=S^{-1}AS$ Consider the matrices below
$$A=\begin{bmatrix}9&4&5\\-4&0&-3\\-6&-4&-2\end{bmatrix}$$
and
$$B=\begin{bmatrix}2&1&0\\0&2&0\\0&0&3\end{bmatrix}$$
These matrices have the same eigenvalues $\{2,2,-3\}$ and the same Jordan Canonical Form so they are similar. 
In trying to find $S$ s.t. $B=S^{-1}AS$ I set
$$S=\begin{bmatrix}a&b&c\\d&e&f\\g&h&k\end{bmatrix}$$
and then tried to solve the system of 9 equations with 9 unknowns 
$$B=S^{-1}AS\Leftrightarrow SB=AS$$
but Matlab showed it is rank deficient so it provided only the zero solution.
How can I find such matrix $S$?
Is there a systematic way to do that in the general case when both $A,B$ are $n\times n$ matrices?
 A: Yes, there is a systematic way. It works as you said. Writing $AS=SB$ as a system of equations in the $n^2$ variables of $S$, we just solve this system of linear equations. In general, there will be several solutions, including $S=0$ of course. But then we impose the condition $\det(S)\neq 0$, which is a polynomial inequality on the variables of $S$. Then we obtain all possible invertible $S$ with $AS=SB$. Again, there are many such $S$ in general.
So, explicitly, solving the linear equations, we obtain
$$
S=\begin{pmatrix} s_1 & s_4 & s_7 \cr -\frac{s_1}{2} & \frac{s_1-2s_4}{4} &
-\frac{2s_7}{3} \cr -s_1 & -s_4 & -\frac{2s_7}{3} \end{pmatrix}
$$
for arbitary parameters $s_1,s_4,s_7$. However,
$$
\det(S)=\frac{s_1^2s_7}{12},
$$
so that we can take all such $S$ with $s_1s_7\neq 0$.
A: Notice that $B$ is precisely the Jordan form of $A$, so it suffices to find a Jordan basis for $A$.
We have:
$$A - 2I =\begin{bmatrix}7&4&5\\-4&-2&-3\\-6&-4&-4\end{bmatrix},\quad (A - 2I)^2 =\begin{bmatrix}3&0&3\\-2&0&-2\\-2&0&-2\end{bmatrix}$$ 
so $\ker (A - 2I)^2 =  \operatorname{span}\{e_1- e_3, e_2\} = \operatorname{span}\left\{\pmatrix{1 \\ 0\\ -1}, \pmatrix{0\\1\\0}\right\}$.
$$A - 3I = \begin{bmatrix}6&4&5\\-4&-1&-3\\-6&-4&-5\end{bmatrix} \implies \ker (A - 3I) = \operatorname{span}\left\{\pmatrix{-3 \\ 2 \\ 2}\right\}$$
Therefore, one Jordan basis is 
$$\left\{(A - 2I)e_2, e_2, \pmatrix{-3 \\ 2 \\ 2}\right\} = \operatorname{span}\left\{\pmatrix{4 \\ -2\\ -4}, \pmatrix{0\\1\\0}, \pmatrix{-3 \\ 2 \\ 2}\right\}$$
So the similarity matrix is 
$$S = \begin{bmatrix}4&0&-3\\-2&1&2\\-4&0&2\end{bmatrix}$$
A: Since $A$ and $B$ have the same Jordan normal form $J$, there exist invertible matrices $P$ and $Q$ such that
$$P^{-1}AP=J=Q^{-1}BQ\text{.}$$
Then
$$B=(PQ^{-1})^{-1}APQ^{-1}\text{.}$$
A: Let $v_1=(-2,1,2)$, $v_2=(-1,0,1)$, and $v_3=(-3,2,2)$. Then $A.v_1=2v_1$, $A.v_2=2v_2+v_1$, and $A.v_3=3v_3$. Therefore, if$$M=\begin{pmatrix}-2&-1&-3\\1&0&2\\2&1&2\end{pmatrix},$$then$$M^{-1}.A.M=\begin{pmatrix}2&1&0\\0&2&0\\0&0&3\end{pmatrix}=B.$$
