If $(AB)^3=BA$, then all matrices commute for a closed set 
Let $\mathcal{M}$ be a subset of $M_n(\mathbb{R})$, the vector space of all $n\times n$ matrices over $\mathbb{R}$. If $AB\in\mathcal{M}$ and $(AB)^3=BA$ hold for all matrices $A, B\in \mathcal{M}$, then
(1) $AB=BA$ for all $A, B\in \mathcal{M}$;
(2) if $I_n\in\mathcal{M}$, then ${\rm det}(A)=\pm 1$ or $0$ for all $A\in\mathcal{M}$.

It seems to have some connections with abstract algebra. The first statement seems similar to proving that for a semi-group $G$, if $(gh)^3=hg$, then $G$ is abelian. The assumption of possessing the identity matrix in the second statement makes it a monoid. Maybe we do not need to apply abstract algebra here, just a thought. But I’m stuck when trying to proceed. Any help is highly appreciated.
 A: Yes, the result in the first part is true if $\mathcal M$ is merely an abstract semi-group, and as pointed out by Arthur in a comment below, the exponent $3$ in the given condition can be replaced by any positive integer $m$. That is, suppose that for some fixed integer $m\ge1$, the following statement is true:
$$
\forall A,B\in\mathcal M, (AB\in\mathcal M\ \wedge\ (AB)^m=BA).
$$
Then we may assert that all member of $\mathcal M$ commute.
Since the case $m=1$ is trivial, we assume $m\ge2$. Let $A,B\in\mathcal M$ and $P=AB$. One may use the given condition to prove inductively that $P,P^2,P^3,\ldots\in\mathcal M$. Therefore
$$
P^{m^2}=(P^{m-1}P)^m=PP^{m-1}=P^m=(AB)^m=BA.
$$
Yet, we also have
$$
P^{m^2}=((AB)^m)^m=(BA)^m=AB.
$$
Hence $AB=BA$. The idea of the proof above is to show that $P^m\,(=P^{m^2})=P$ whenever $P$ is a product of two members of $\mathcal M$.
A: Okay, so I was reading it wrong in the comments and the problem is much simpler then (unless I am grossly mistaken somewhere). If $\mathcal{M}$ has only one element, then there is nothing to prove. Otherwise, take $A\in\mathcal{M}\implies A^2\in\mathcal{M}$ and so $A^6 = A^2.$ Then again $A^3 = AA^2\in\mathcal{M}\implies A^9 = A^3.$ So the minimal polynomial of $A$ must divide both $x^2(x^2-1)(x^2+1)$ and $x^3(x^2-1)(x^4+x^2+1)$, which means the it also divides $x^2(x^2-1).$ Therefore, $\forall A\in\mathcal{M}\implies A^4 = A^2.$
Now:
$$AB^2A =AB(AB)^3 = (AB)^4 = (AB)^2 = (AB)^4 = (AB)^3(AB)=BA^2B$$
and by symmetry:
$$BA^2B = (BA)^2 = AB^2A.$$ Furthermore:
$$(AB)^2 = (AB)^4 = ABABABAB = A(BA)^3B = A^2B^2$$ and again similarly:
$$(BA)^2 = B^2A^2.$$
All of this conclude:
$$A^2B^2 = (AB)^2 = BA^2B = (BA)^2 = B^2A^2.$$
We are almost there and to finish:
$$AB = (BA)^3 = (BA)^2(BA) = (AB)^2(AB) = (AB)^3 = BA.$$
