Prove $n$ is even and $\text{rank}(AB)^{k}=n/2$ Let $n\geq2$ be an integer and $A, B$ two $n\times n$ matrices with complex entries, such that $A^{2}=B^{2}=0$, with $A+B$ being invertible. Prove that $n$ is even and $\text{rank}(AB)^{k}=n/2,$ for all $k\geq2.$
 A: To begin with, here is a proof that $\operatorname{rank}(A) = \operatorname{rank}(B) = \frac n2$, which incidentally proves that $n$ is even.
If $\operatorname{rank}(A)=r$, then there exist $r$ independent vectors $\{Av_1, Av_2, \dots, Av_r\}$. We know $A^2v_1 = A^2v_2 = \dots = A^2v_r = 0$, because $A^2=0$, so all $r$ of these vectors are in the null space of $A$, and we have $\dim \operatorname{Null}(A) \ge r$. Therefore we have $$2r \le r + \dim \operatorname{Null}(A) = n \implies r \le \frac n2$$ by applying the rank-nullity theorem. Similarly, we get $\operatorname{rank}(B) \le \frac n2$.
To prove the reverse inequality, note that $\operatorname{rank}(A+B) \le \operatorname{rank}(A) + \operatorname{rank}(B)$ (this is true for any two matrices $A$ and $B$). But $A+B$ is invertible, so it has rank $n$. Therefore
$$n = \operatorname{rank}(A+B) \le \operatorname{rank}(A) + \operatorname{rank}(B) \le \operatorname{rank}(A) + \frac n2 \implies \operatorname{rank}(A) \ge \frac n2$$ and we get $\operatorname{rank}(B) \ge \frac n2$ in the same way.

The proof that $\operatorname{rank}((AB)^k) = \frac n2$ follows more or less the same logic.
To get the upper bound, that $\operatorname{rank}((AB)^k) \le \frac n2$, suppose that $\operatorname{rank}((AB)^k) = r$ and there are $r$ independent vectors $\{(AB)^k w_1, (AB)^k w_2, \dots, (AB)^k w_r\}$. Once again, all of these are in the null space of $A$ (since if you multiply by $A$ on the left, the product begins with $A^2$), so there can be at most $\frac n2$ of them. Similarly, $\operatorname{rank}((BA)^k) \le \frac n2$: we'll need this, too.
Inductively, we can show that for all $k\ge 1$, $$(A+B)^{2k} = (AB)^k + (BA)^k.$$ When $k=1$, we have $(A+B)^2 = A^2 + AB + BA + B^2$, and $A^2=B^2=0$ so those terms cancel. To go from $k$ to $k+1$: 
\begin{align}
(A+B)^{2(k+1)} &= (A+B)^2 ((AB)^k + (BA)^k) \\
 &= (AB + BA)((AB)^k + (BA))^k \\
 &= (AB)^{k+1} + AB(BA)^k + BA(AB)^k + (BA)^{k+1} \\
 &= (AB)^{k+1} + (BA)^{k+1}
\end{align}
where the $BA(AB)^k$ and $AB(BA)^k$ terms cancel due to having an $A^2$ or $B^2$ factor.
Since $\operatorname{rank}(A+B)=n$, we have $\operatorname{rank}((A+B)^k) = n$ for all $k$, so 
$$n = \operatorname{rank}((AB)^k + (BA)^k) \le \operatorname{rank}((AB)^k) + \operatorname{rank}((BA)^k)$$
and, as before, since both ranks on the right are at most $\frac n2$, the inequality can only hold if both are exactly $\frac n2$.
