Solution of $\dot{X}(t) = A X(t)B $ Let $A, B, X_0 \in M_{n, n}(\mathbb{R})$. What is the solution to the following differential equation ?
$
\left\{
    \begin{array}{ll}
        X(0) = X_0 \\
        \dot{X}(t) = A \cdot X(t) \cdot B
    \end{array}
\right.
$
If $A, B$ and $X_0$ commute, we find easily $X(t) = X_0 \exp\left(A\cdot B\;t\right)$ but in a more general setup I didn't find anything else.
 A: $$X(t)=\sum_{n\ge 0}\frac{t^n}{n!} A^n X_0 B^n$$
Not the general case but assume that $A,B$ are diagonalizable. Take an eigenvector $B v = \lambda v$, decompose $X_0v$ in the basis of $A$'s eigenvectors that is
$X_0 v=\sum_{j=1}^n w_j,A w_j= \ell_j w_j$. Then $$X(t) v =
\sum_{n\ge 0}\frac{t^n}{n!} A^n X_0 B^n v=
\sum_{n\ge 0}\frac{t^n}{n!} A^n X_0 \lambda^n v  =  \sum_{n\ge 0}\frac{\lambda^n t^n}{n!}A^n \sum_{j=1}^n w_j $$ $$ = \sum_{n\ge 0}\frac{ \lambda^n t^n}{n!} \sum_{j=1}^n \ell_j^n w_j = 
\sum_{j=1}^n \exp(\lambda \ell_j t) w_j
$$

Without assuming that $A,B$ are diagonalizable:
For $v$ an eigenvector of $B$, $A$ doesn't need to be diagonalizable if we are satisfied with $X(t)v=\exp(\lambda A t) X_0v$. Then if $B\in M_k(\Bbb{C}) $ is not diaongalizable we can write $B=P (\Lambda+N) P^{-1}$ with $\Lambda$ diagonal, $N^k=0$ and $N\Lambda=\Lambda N$. Then $B^n P e_j = P\sum_{m=0}^{k-1} {n \choose m}\Lambda_{jj}^{n-m} N^m e_j$ so $$X(t)Pe_j = 
\sum_{n\ge 0}t^n A^n \sum_{m=0}^{\min(k-1,n)}\frac1{m! (n-m)!} \Lambda_{jj}^{n-m} X_0 N^m P e_j$$ $$=
\sum_{m=0}^{k-1} t^m A^m \exp(A \Lambda_{jj} t)\frac1{m!} X_0 P N^m e_j$$
A: There is no closed-form solution in terms of elementary functions for this matrix-valued differential equation. Assuming that both $A$ and $B$ are diagonalizable with diagonal matrices $D_A$ and $D_B$, respectively, one has
$$
\dot{Y}(t)=D_AY(t)D_B
$$
where $Y(t)=PX(t)Q^{-1}$, $D_A=PAP^{-1}$ and $D_B=QBQ^{-1}$. From there, one can use the ideas proposed in reuns' answer to obtain
$$
X(t)=\sum_{n\ge0}\dfrac{t^n}{n!}X_0\circ a_nb_n^T
$$
where $a_n,b_n$ are vectors containing the diagonal entries of $D_A^n$ and $D_B^n$, respectively.
We can also use vectorization to get
$$
\mathrm{vec}(\dot{X}(t))=(B^T\otimes A)\mathrm{vec}(X(t)),
$$
which leads to
$$ 
\mathrm{vec}(X(t))=\exp((B^T\otimes A)t)\mathrm{vec}(X(0)).
$$
A: I know that reuns' answer has already been accepted, but here is a derivation (too long for a comment) of his solution, which doesn't need any assumptions on $A$ and $B$.
After integrating directly the original differential equation, i.e.
$$
X(t)-X_0 = \int_{X_0}^{X(t)}\mathrm{d}X = \int_0^t AX(s)B\,\mathrm{d}s,
$$
one understands that the solution can be expanded with the help of this recursive formula; indeed, we get :
$$
\begin{array}{rcl}
X(t) 
   &=& \displaystyle
   X_0 + \int_0^t AX(s)B\,\mathrm{d}s \\
   &=& \displaystyle
   X_0 + \int_0^t A\left(X_0 + \int_0^s AX(s')B\,\mathrm{d}s'\right)B\,\mathrm{d}s \\
   &=& \displaystyle
   X_0 + \int_0^t AX_0B\,\mathrm{d}s + \int_0^t\mathrm{d}s_1\int_0^{s_1}\mathrm{d}s_2 \,A^2X(s_2)B^2 \\
   &=& \displaystyle
   \ldots \\
   &=& \displaystyle
   \sum_{n=0}^\infty \int_0^t\mathrm{d}s_1 \int_0^{s_1}\mathrm{d}s_2 \cdots \int_0^{s_{n-1}}\mathrm{d}s_n \,A^nX_0B^n \\
   &=& \displaystyle
   \sum_{n=0}^\infty \frac{t^n}{n!} A^nX_0B^n
\end{array}
$$
