What is an example of a positive integer $n$ and two nilpotent $n\times n$ matrices $A$ and $B$ such that $AB$ is NOT nilpotent?
2 Answers
$\begingroup$
$\endgroup$
Try $$\begin{pmatrix} 0 & 1 \\ 0& 0 \end{pmatrix}$$ and $$\begin{pmatrix} 0 & 0 \\ 1& 0 \end{pmatrix}.$$ Now compute $$\begin{pmatrix} 0 & 1 \\ 0& 0 \end{pmatrix}\cdot \begin{pmatrix} 0 & 0 \\ 1& 0 \end{pmatrix} =\begin{pmatrix} 1 & 0 \\ 0& 0 \end{pmatrix} ,$$ which is not nilpotent.
$\begingroup$
$\endgroup$
In general, take any two $n \times n$ matrices $A$ and $B$ where $Av_k = v_{k+1}$ for particular vectors $v_1, \ldots, v_n$ where $v_1,\ldots,v_{n-1}$ are linearly independent and $v_n = 0$, and similarly where $Bw_k = w_{k+1}$ for vectors $w_1,\ldots,w_n$ where $w_1,\ldots,w_{n-1}$ are linearly independent and $w_n = 0$. Then $A$ and $B$ will be nilpotent. However, if you set $v_1 = w_2$ and $v_2 = w_1$, then $AB$ will map $w_1$ to $v_1$ and will map $v_1$ to $w_1$, so it cannot be nilpotent.
