On the nilpotence of the matrix $AB-BA$ Given $n\times n$ matrices $A,B$ satisfy:
$rank(AB-BA)=1$
Prove that $(AB-BA)^{2}=0$
Generalize the problem if possible.
Any solution not mention Jordan canonical form would be appreciated!
 A: Hint: show that $\text{tr}(AB - BA) = 0$.  Use this fact and the rank to find the Jordan Canonical form of $AB-BA$.
The generalization they want might be "what if $\text{rank}(AB-BA) = r$"?

Suppose that $AB - BA = UTU^*$ with $T$ upper triangular.  It follows that $T$ has a rank of $1$ and a trace of $0$.
Since $T$ has rank $1$, it has at most one non-zero diagonal entry.  However, if $T$ has a single non-zero entry on the diagonal, then it would have a non-zero trace.
So, $T$ is a strictly upper triangular matrix with rank $1$. In particular, $T$ has the form
$$
T = 
\pmatrix{
0&*&*\\
&\ddots&*\\
&&0
}
$$
Note that the rank of $T^2$ must be strictly less than the rank of $T$. It follows that $T^2$ has rank $0$, so that it must be the zero matrix.
A: Let $e_j, j=1,\cdots,n$ be the canonical basis of $\mathbb{R}^n$. The rank of $M$ is $1$. Hence there exists $a_j\in \mathbb{R}$, not all zero, and a vector $v=(v_1,\cdots,v_n)$, not zero, such that $Me_j=a_jv$ for all $j=1,\cdots,n$. We immediately have that the trace of $M$ is $T=a_1v_1+a_2v_2+\cdots+a_n v_n$. Hence $T=0$.
Now, for $j=1,\cdots,n$:
$$M^2e_j=a_jMv= a_jM(\sum v_k e_k)=a_j(\sum v_k a_k v)=a_jTv=0$$
Hence $M^2=0$.
