# $\operatorname{rank}(A^2)+\operatorname{rank}(B^2)\geq2\operatorname{rank}(AB)$ whenever $AB=BA$?

Let $$A,B$$ be $$n\times n$$ matrices. If $$AB=BA$$, then $$\operatorname{rank}(A^2)+\operatorname{rank}(B^2)\geq2\operatorname{rank}(AB)$$.

Is this rank inequality correct? No counterexample seems to exist. Here's what I've done. When $$A$$ is a Jordan block this is easily proved. I tried to bring $$A$$ into Jordan form for $$n=2,3,4$$ and found no counterexample. So currently I think it is true. By Fitting's lemma it suffices to consider the case in which $$A$$ is nilpotent. We can bring $$A$$ into Jordan form. Then the blocks in $$B$$ are upper triangular. However, even in this case $$\operatorname{rank}(B^2)$$ is not tractable.

Any hints will be appreciated!

• I don't know but it's funny, that this looks very much alike $${\rm rank}(A^2 + B^2 - 2AB) = {\rm rank}((A-B)^2) \geq 0$$ if $AB=BA$ ;) . – Diger Nov 26 '18 at 13:57
• What about $${\rm rank}(A^2 + B^2 - 2AB) \leq {\rm rank}(A^2) + {\rm rank}(B^2) + {\rm rank}(-2AB) \, ?$$ – Diger Nov 26 '18 at 14:36
• @Diger You got me! It was inspired by the famous inequality $a^2+b^2\geq2ab$. :) – Colescu Nov 26 '18 at 14:43

Well, turns out I made a mistake when trying to construct counterexamples... There is indeed a counterexample of order $$4$$: $$A=\begin{pmatrix}0&1&0&0\\0&0&0&0\\0&0&0&1\\0&0&0&0\end{pmatrix},\quad B=\begin{pmatrix}0&1&1&0\\0&0&0&1\\0&0&0&1\\0&0&0&0\end{pmatrix}.$$ Sorry for misleading. :(
• Now as you have presented counterexample it seems very logical: rank$(A^2)=0$, rank$(B^2)=1$ and if only $AB=BA \ne 0$ then counterexample is ready.. – Widawensen Nov 27 '18 at 13:05
• We see that the counterexample is of the form $AB=\begin{bmatrix}N & I \\ 0 & N \end{bmatrix}\begin{bmatrix}N & 0 \\ 0 & N \end{bmatrix}=\begin{bmatrix}0 & N \\ 0 & 0 \end{bmatrix}=BA$ with $\begin{bmatrix}N & I \\ 0 & N \end{bmatrix}^2=\begin{bmatrix}0 & 2N \\ 0 & 0 \end{bmatrix}$ and $\begin{bmatrix}N & 0 \\ 0 & N \end{bmatrix}^2=\begin{bmatrix}0 & 0 \\ 0 & 0 \end{bmatrix}$. Maybe it's worth to memorize for future questions :) .. Thank you for the inspiring question. – Widawensen Nov 27 '18 at 14:12