# Two invertible matrices

Let $A,B$ be two $n\times n$ invertible matrices with complex entries. Also, let $\alpha, \beta \in \mathbb{C}$ with $|\alpha| \neq |\beta|$ such that $\alpha AB+\beta BA=I_n$. Prove that $\det(AB-BA)=0$.

I tried to manipulate the given equation in order two get $(AB-BA)$ as a factor somewhere, but didn't manage to get anything useful. I also thought of using $A^{-1}$ and $B^{-1}$ somewhere, but I only got messier relations.

I leave my first answer below. Here is a much easier one: As below, we may assume $AB + \gamma BA = I$, where $|\gamma|\neq 1$ and $\gamma\neq 0$. Put $\lambda_0 = (1+\gamma)^{-1}$. Then $$AB-\lambda_0 = 1 - \lambda_0 - \gamma BA = -\gamma\left(BA - \frac{1-\lambda_0}{\gamma}\right) = -\gamma(BA-\lambda_0).$$ Hence, as $AB$ and $BA$ have the same eigenvalues, $$\sigma(BA-\lambda_0) = \sigma(AB-\lambda_0) = -\gamma\cdot\sigma(BA-\lambda_0).$$ Thus, multiplication by $(-\gamma)$ leaves the finite set $\sigma(BA-\lambda_0)$ invariant. But, as $|\gamma|\neq 1$, this is only possible if $\sigma(BA-\lambda_0) = \{0\}$. Hence, $BA-\lambda_0$ is nilpotent. And as $$AB-BA = I - \gamma BA - BA = I - \lambda_0^{-1}BA = -\lambda_0^{-1}(BA - \lambda_0),$$ the same holds for $AB-BA$. In particular, $AB-BA$ is not invertible, i.e., $\det(AB-BA)=0$.

The statement is clear for $\alpha = 0$. Hence, let $\alpha\neq 0$. In this case, with $A' = \alpha A$ we have $A'B + \frac{\beta}{\alpha}BA' = I$. Hence, we may assume that $AB + \gamma BA = I$ with $|\gamma|\neq 1$ and $\gamma\neq 0$.

Let $x$ be an eigenvector of $AB$ with respect to the eigenvalue $\lambda$. Then $$\lambda x + \gamma BAx = x,$$ that is, $$BAx = \frac{1-\lambda}\gamma x.$$ But we know that $AB$ and $BA$ have exactly the same eigenvalues (even the same Jordan structures) as they are both invertible. Hence, the function $f(z) = \tfrac{1-z}\gamma$ is a selfmap of the set of eigenvalues. Therefore, there is an eigenvalue $\lambda$ such that $f^n(\lambda) = \lambda$ for some $n$, where $f^n = f\circ\ldots\circ f$ ($n$ times).

One can easily prove by induction that $$f^n(z) = \frac{1 - (-\gamma^{-1})^n}{1+\gamma} + (-\gamma^{-1})^nz$$ and then (since $(-\gamma^{-1})^n\neq 1$) that the only fixed point of each $f^n$ is $z = (1+\gamma)^{-1}$. Thus, $\lambda = (1+\gamma)^{-1}$. In particular, $f(\lambda) = \lambda$. But then, with the eigenvector $x$ from above, we have $$(BA-AB)x = BAx - ABx = f(\lambda)x - \lambda x = 0.$$ Hence, the matrix $BA-AB$ is not invertible, meaning that $\det(BA-AB) = 0$.

• your proof is incomplete. In particular, you don't use the essential condition $|\alpha|\not= |\beta|$. – loup blanc Oct 10 '17 at 1:36
• In other words, when you write " (here, the condition on $\alpha$ and $\beta$ comes into play)" you prove nothing... – loup blanc Oct 10 '17 at 1:41
• Isn't it a little bit hard to write "you prove nothing" when you are not capable of proving that each $f^n$ has only one fixed point? This place gets more and more crowded with offensive people... – amsmath Oct 10 '17 at 9:41
• I edited my answer so that also loup blanc can find the fixed points of $f^n$. – amsmath Oct 10 '17 at 11:55
• You are pretentious and you did not understand. The key of proof is to say "for every $n$, $(-\gamma^{-1})^n$ cannot be $1$ because $|\gamma|\not= 1$". – loup blanc Oct 10 '17 at 12:21

I eventually managed to find a solution, different from @amsmath's one. From the given equation we get $$AB=\frac{1}{\alpha}I_n-\frac{\beta}{\alpha}BA \quad (1)$$ and also $$\alpha B+\beta A^{-1}BA=A^{-1} \quad (2)$$ hence $$d=\det(AB-BA)=\det \left(\frac{1}{\alpha}I_n-\frac{\alpha+\beta}{\alpha}BA \right)=\frac{1}{\alpha^n}\det(I_n-(\alpha+\beta)BA)$$ But $A^{-1}A=I_n$ so $d=\frac{1}{\alpha^n}\det(A^{-1}A-(\alpha+\beta)BA)=\frac{1}{\alpha^n}\det A \det(A^{-1}-(\alpha+\beta)B)$ and using $(2)$ we'll have $$d=\frac{1}{\alpha^n}\det A \det(\beta A^{-1}BA-\beta B)=\frac{\beta^n}{\alpha^n}\det A \det(A^{-1}BA-B)=\frac{\beta^n}{\alpha^n}\det(BA-AB)$$ Finally, $d=\det(AB-BA)=\frac{\beta^n}{\alpha^n}(-1)^n\det(AB-BA)$ and since $|\alpha| \neq |\beta|$, the conclusion follows.

• Your proof is correct but, in fact, the hypothesis "$A,B$ invertible" is useless. – loup blanc Oct 10 '17 at 2:39
• It is not useless, but redundant, since it follows from the hypothesis. – amsmath Oct 10 '17 at 13:22

As loup blanc has pointed out in a comment, the assumption that $A$ and $B$ are invertible is useless. Even the assumption that $|\alpha|\ne|\beta|$ is too strong. As we will see shortly, all we need is that $\alpha^n\ne(-\beta)^n$.

From the condition that $\alpha AB+\beta BA=I$, we get \begin{align} \alpha(AB-BA) &= (I-\beta BA)-\alpha BA = I - (\alpha+\beta)BA,\\ -\beta(AB-BA) &= -\beta AB + (I-\alpha AB) = I - (\alpha+\beta)AB. \end{align} Since $A$ and $B$ are square matrices, $AB$ and $BA$ have identical spectra. It follows that $\alpha^n\det(AB-BA) = (-\beta)^n \det(AB-BA)$. Therefore, when $\alpha^n\ne(-\beta)^n$, we must have $\det(AB-BA)=0$.