$A^2 - B^2$ is an invertible matrix and $A^5 = B^5$ and $A^3B^2 = A^2B^3$ . Then what is the determinant of the matrix $A^3 + B^3$? $A,B$ are two $3 \times 3$ real-matrices with following three properties.

*

*$A^2 - B^2$ is an invertible matrix

*$A^5 = B^5$

*$A^3B^2 = A^2B^3$
Then what is the determinant of the matrix $A^3 + B^3$ ?
 A: $A$ and $B$ are real $3\times3$ matrices, so their eigenvalue polynomials are cubic and hence have at least one real zero. Let $\lambda\in\Bbb{R}$ be an eigenvalue of $B$, and $x\in\Bbb{R}^3$ be the corresponding eigenvector. Then
$$
A^5x=B^5x=\lambda^5x.\tag{1}
$$
We also have
$$
\lambda^3A^2x=(A^2B^3)x=A^3B^2x=\lambda^2A^3x.\tag{2}
$$
Apply $A^3$ to both sides of $(2)$, and use $(1)$ to arrive at
$$
\lambda^8x=A^3(\lambda^3A^2x)=A^3(\lambda^2A^3x)=\lambda^2A^6x=\lambda^7Ax.\tag{3}
$$
If $\lambda\neq0$ then $(3)$ implies that $Ax=\lambda x$ also. This violates the first assumption as then
$$(A^2-B^2)x=\lambda^2x-\lambda^2x=0.$$
We are left with the case $\lambda=0$. Again, by $(1)$ we see that $A^5x=0$. Because $A$ is a $3\times3$ matrix this implies (think Jordan canonical form) that already $A^3x=0$.
We have shown that $(A^3+B^3)x=0$ for a non-zero vector $x$, and can conclude that $\det(A^3+B^3)=0$.
A: Let $V=\mathbb R^3$. The condition $A^5=B^5$ implies that $A^5V=B^5V$. Since $A$ is $3\times 3$, the descending chain $V\supseteq AV\supseteq A^2V\supseteq\cdots$ must have been stabilised at $A^3V$ and the similar is true for $B^3V$. Therefore from $A^5V=B^5V$ we get $W:=A^3V=B^3V$.
Furthermore, as the descending chains have been stabilised, both $A$ and $B$ are automorphisms on the invariant subspace $W$. The condition $A^3B^2=A^2B^3$ thus implies that $A=B$ on $W$. As $A^2-B^2$ is invertible, $W$ must be zero. Hence $A^3V=B^3V=0$, i.e., $A^3=B^3=0$. Consequently $\det(A^3+B^3)=0$.
Remark. Actually, for any number $c$, “$\det(A^3+B^3)=c$” is a correct answer, because no matrices $A$ and $B$ can satisfy all three given conditions in the first place. Suppose the contrary. By our previous argument, $A$ and $B$ must be nilpotent. Since the matrices are $3\times3$, the ranks of $A^2$ and $B^2$ are at most one. Therefore the rank of $A^2-B^2$ is at most two. Hence $A^2-B^2$ is singular, but this is a violation of the first condition.
However, if we change the first condition to “$A-B$ is invertible”, there are feasible pairs of $A$ and $B$, such as
$$
A=\pmatrix{0&2&0\\ 0&0&1\\ 0&0&0}
\text{ and }B=\pmatrix{0&1&0\\ 0&0&0\\ 1&0&0}.
$$
In this case, our previous argument still applies and we may conclude that $A^3=B^3=0$ and $\det(A^3+B^3)=0$.
