Is it always true that $\det(A^2+B^2)\geq0$?

Question

Let $A$ and $B$ be real square matrices of the same size. Is it true that $$\det(A^2+B^2)\geq0\,?$$

If $AB=BA$ then the answer is positive: $$\det(A^2+B^2)=\det(A+iB)\det(A-iB)=\det(A+iB)\overline{\det(A+iB)}\geq0.$$

Answer 1

If $A= \left( \begin{matrix} 1 &1 \\ 0 &1 \end{matrix} \right)$ and $B=\left( \begin{matrix} 1&0 \\ n&1 \end{matrix} \right)$, then $\det(A^2+B^2)=4(1-n)$.

Answer 2

The ideas in Seirios' answer and Jyrki Lahtonen's (now deleted) answer can be generalised to the following result:

Let $n\ge2$ and $A\in M_n(\mathbb{C})$. Then $\det(A^2+B^2)\ge0$ for all $B\in M_n(\mathbb{R})$ if and only if $A^2$ is a nonnegative scalar multiple of $I_n$.

Proof.

("$\Leftarrow$") Suppose $A^2=pI$ for some $p\ge0$. Since nonreal eigenvalues of $B$ occur in conjugate pairs and the squares of the real eigenvalues of $B$ are nonnegative, it follows that $\det(A^2+B^2)\ge0$.

("$\Rightarrow$") We first show that $A^2$ is real. Let $C=A^2=(c_{ij})$ and $B^2=\operatorname{diag}(0,\lambda,\lambda,\ldots,\lambda)$, where $\lambda>0$. By the given condition, $\det(A^2+B^2)=c_{11}\lambda^{n-1}+o(\lambda^{n-1})\ge0$ for all sufficiently large $\lambda$. Hence $c_{11}$ must be real. Similarly, other entries of $C$ are real too, i.e. $A^2$ has to be real.

It remains to show that:

If $n\ge2$ and $A^2\in M_n(\mathbb{R})$ is not a nonnegative multiple of $I_n$, then there exists $B\in M_n(\mathbb{R})$ such that $\det(A^2+B^2)<0$.

Consider the case $n=2$ first. Let $R(\theta)=\pmatrix{\cos\theta&\sin\theta\\ -\sin\theta&\cos\theta}$. We may assume that $A^2$ is in its real Jordan form. There are four possibilities:

1. $A^2=\pmatrix{\lambda&1\\ 0&\lambda}$ for some $\lambda\in\mathbb{R}$. Then $\det(A^2+B^2)<0$ when $B=B_0:=\pmatrix{1&0\\ b&1}$ for sufficiently large $b$.
2. $A^2=\pmatrix{\lambda_1&0\\ 0&\lambda_2}$ for distinct $\lambda_1,\lambda_2\in\mathbb{R}$. Let $B=bR(\frac\pi2)$. Then $\det(A^2+B^2)=(\lambda_1^2-b^2)(\lambda_2^2-b^2)<0$ when $b^2$ is strictly between $\lambda_1^2$ and $\lambda_2^2$.
3. $A^2=a^2R(\theta)$ with $a\neq0$ and $\sin\theta\neq0$. If $\sin\theta>0$, let $B=aB_0$. Then $\det(A^2+B^2)=2a^2(1+\cos\theta) - 2a^2b^2\sin\theta<0$ when $b$ is large enough. If $\sin\theta<0$, choose $B=aB_0^T$ instead.
4. $A^2=-a^2I_2$ with $a\neq0$. Let $B=\pmatrix{\lambda_1&0\\ 0&\lambda_2}$. Then $\det(A^2+B^2)=(\lambda_1^2-a^2)(\lambda_2^2-a^2)<0$ when $\lambda_1^2,\lambda_2^2$ bracket $a^2$.

Now, if $n>2$, we may assume that some diagonal block $\widetilde{A}$ in the real Jordan form of $A$ has one of the above forms. Therefore, if $B$ is a block-diagonal matrix whose corresponding diagonal block to $\widetilde{A}$ is chosen as in the above and each of the other diagonal blocks is $1\times1$ and is equal to $\rho(A)+1$, then $\det(A^2+B^2)<0$. (The addition of $1$ to $\rho(A)$ is needed to guarantee that the determinant is nonzero and the sign of the determinant is solely modified by the diagonal block corresponding to $\widetilde{A}$.)