Find all matrices $X$ such that $ABXB^tA^t=I$ Find all matrices $X$ such that: 
$$ABXB^tA^t=I$$ if 
$A=\begin{pmatrix}
1 &-2 &2\\
3 &-5 &6\\
-1 &2 &-1
\end{pmatrix}$ and $B=\begin{pmatrix}
-3 &-2 &-2\\
2 & 1 &1\\
6 &3 &4
\end{pmatrix}$. 
So I managed to get that $AB=\begin{pmatrix}
5 &2 &4\\
17 &7 &13\\
1&1&0
\end{pmatrix}$ and  $B^tA^t=(AB)^t=\begin{pmatrix}
5 &17 &1\\
2 &7 &1\\
4 &13 &0
\end{pmatrix}$
So we have $\begin{pmatrix} 
5 &2 &4\\
17 &7 &13\\
1&1 &0
\end{pmatrix}\cdot X\cdot\begin{pmatrix}
5 &17 &1\\
2&7&1\\
4&13&0
\end{pmatrix}=I$  
Now how do I get $X$ from here?
 A: If either $AB$ or $B^TA^T$ is singular, there is no chance to get a solution. 
Therefore the $3 \times 3$ matrices, have to be nonsingular (and they are, I just checked). 
So, to get $X$ you need to invert both of them and get to $X=B^{-1}A^{-1}A^{-T}B^{-T} = B^{-1}(A^TA)^{-1}B^{-T}$
Usually you would decompose $AB$ into $LR$ or $QR$ decompositions and pull them to the right hand side. 
A: From $ABXB^tA^t=I$, we see that $det(A) \ne 0 \ne det(B)$. Hence $A$ and $B$ are invertible.
Again , from $ABXB^tA^t=I$, we see
$X=(AB)^{-1}((AB)^{-1})^t$
A: Matrices multiplication is not commutative. You are allowed to multiply an equation by a non-zero matrix, but you have to mind the side from which you are applying the multiplication.
Let us say that $P=AB$ and $Q=B^TA^T$.
Your equation takes the form of $P \cdot X \cdot Q = I$.
You can multiply each side by $P^{-1}$ from the left and $Q^{-1}$ from the right to get (note that $\det(P)\ne0\ne\det(Q)$, which makes $P$ and $Q$ invertible):
$A^{-1} \cdot P \cdot X \cdot Q \cdot Q^{-1} = P^{-1} \cdot I \cdot Q^{-1}$
This can be simplified based on $M^{-1} \cdot M = M \cdot M^{-1} = I$:
$I \cdot X \cdot I = P^{-1} \cdot I \cdot Q^{-1}$
This can further be simplified based on $M \cdot I = M$:
$X=P^{-1} \cdot Q^{-1}$
Mind that:
$M^{-1}=\frac{1}{\det(M)}\left(M^*\right)^T$
Where $M^*$ is the cofactor matrix.
