Proving that determinant is zero

Let $$A,B \in M_3(\mathbb{C})$$ such that $$(AB)^2 = A^2B^2$$ and $$(BA)^2 = B^2A^2$$. Prove that $$\det(AB-BA) = 0$$.

• What have you tried? – CiaPan Apr 4 at 9:43
• I tried to prove that rank(AB - BA) is less than 3. – Ddang Apr 4 at 9:46
• And this could work perhaps like here. – Dietrich Burde Apr 4 at 9:47
• I don’t think so. If C = AB - BA, we get from the conditions that ACB = 0 = BCA. I don’t see how to go on from here. Maybe the rank doesn’t matter at all... – Ddang Apr 4 at 9:53

Hints. Let $$C=AB-BA$$. The given conditions imply that $$ACB=0$$ and $$BCA=0$$. We don't need both of them. One --- say, $$ACB=0$$ --- is enough:
• If at least one of $$A$$ or $$B$$ has rank $$\le1$$, argue that $$\operatorname{rank}(C)=\operatorname{rank}(AB-BA)\le2$$.
• If $$\operatorname{rank}(A),\operatorname{rank}(B)\ge2$$, then $$\operatorname{rank}(CB)\ge2$$ when $$C$$ is non-singular. Now consider Sylvester's rank inequality $$\operatorname{rank}(A)+\operatorname{rank}(CB)-3\le\operatorname{rank}(ACB)$$.