# Prove that $\det(AB-BA)=0$

Let $A,B$ be two $3 \times 3$ matrices with complex entries such that $$(A-B)^2=O_3$$ Prove that $$\det(AB-BA)=0$$

I tried to prove this with ranks. I denoted $X=A-B$ and thus $X^2=O_3$ which means that $\det X=0$ and $\operatorname{rank}X \leq 2$. Then, I wrote $AB-BA=(X-B)B-B(X-B)=XB-BX$ and finally I used $\operatorname{rank}(M \pm N) \leq \operatorname{rank}M+\operatorname{rank}N$ and Frobenius's inequality in order to get $$\operatorname{rank}(XB-BX) \leq \operatorname{rank}BX+\operatorname{rank}XB \leq \operatorname{rank}X+\operatorname{rank}BXB$$ and if we knew that $\operatorname{rank}BXB=0$, the problem would be solved. However, I don't quite know if the latter is true.

• Does $O_3$ denote the zero matrix of dimension $3 \times 3$? – md2perpe Dec 26 '17 at 21:49
• I reckon that $\text{rank }X\le1$. – Lord Shark the Unknown Dec 26 '17 at 21:50
• @md2perpe yes, that's it! – AndrewC Dec 26 '17 at 21:51
• Oh yes indeed, it follows from Sylvester's inequality: $\operatorname{rank}X^2=0 \geq 2\operatorname{rank}X-3$ and thus $\operatorname{rank}X \leq 1$. The problem is then solved since $\operatorname{rank}BXB \leq \operatorname{rank}X \leq 1.$ – AndrewC Dec 26 '17 at 21:59
• From $X=A-B$ we get $A=X+B$. You wrote $AB-BA=(X-B)B-B(X-B)$, which needs to be $AB-BA=(X+B)B-B(X+B)$. – user236182 Dec 26 '17 at 22:32

Here is a more or less direct, less creative solution. Since $(A-B)^2=0$, then either $A-B=0$ (in which case $AB-BA=0$), or its Jordan form is $$J=\begin{bmatrix} 0&1&0\\0&0&0\\0&0&0\end{bmatrix}.$$ So $A-B=SJS^{-1}$ for some $S$. Let $A'=S^{-1}AS$, $B'=S^{-1}BS$. Then $A'=B'+J$, and $$A'B'-B'A'=(B'+J)B'-B'(B'+J)=JB'-B'J.$$ Now check directly that $$JB'-B'J=\begin{bmatrix}B'_{31}&B'_{32}&B'_{33}-B'_{11}\\ 0&0&-B'_{21}\\ 0&0&-B'_{31} \end{bmatrix}.$$ Thus $\det(JB'-B'J)=0$. Finally, \begin{align} \det(AB-BA)&=\det(SA'S^{-1}SB'S^{-1}-SB'S^{-1}SA'S^{-1})\\ \ \\ &=\det(A'B'-B'A')=\det(JB'-B'J)=0. \end{align}
• Thank you! But why has $J$ the form you mentioned? I am sorry, but I barely know about Jordan forms at the moment. Could you please provide a good source where I could learn about them? – AndrewC Dec 27 '17 at 9:36
• I think I see now: is it because $X^2$ would be the minimal polynomial of $A-B$ and since $0$ is its only eigenvalue, it means that the size of its largest Jordanian block is $2$? – AndrewC Dec 27 '17 at 10:30
• The largest Jordan block has size 2, because if it had size 3 you would have $J^2\ne0$. And if it had size 1 you would have $J=0$. – Martin Argerami Dec 27 '17 at 11:21
As pointed above by @Lord Shark the Unknown (whose comment struck me, pointing the right way) we have from Sylvester's inequality: $$0=\operatorname{rank}O_3=\operatorname{rank}(X\cdot X) \geq \operatorname{rank}X+\operatorname{rank}X-3 \Rightarrow \operatorname{rank}X \leq 1$$ Thus going back to my last inequality, $$\operatorname{rank}(XB-BX) \leq \operatorname{rank}X+\operatorname{rank}BXB \leq \operatorname{rank}X+\operatorname{rank}X \leq 2$$ and so $\det(AB-BA)=0$.
• Since $\text{rank}X \leq 1$, you do not really need the inequality in red $$\operatorname{rank}(XB-BX) \leq {\color{red}{\operatorname{rank}X+\operatorname{rank}BXB} \leq \operatorname{rank}X+\operatorname{rank}X} \leq 2.$$ Instead,$$\operatorname{rank}(XB-BX) \leq {\color{red}{\operatorname{rank}XB+\operatorname{rank}BX} \leq \operatorname{rank}X+\operatorname{rank}X} \leq 2.$$ – clark Dec 26 '17 at 22:25