Show that equation $\det(A+xB)=0$ has real solutions if and only if $\det(A^{2}+B^{2})\geq(\det(A)+\det(B))^{2}$ We have $A,B$ two $2×2$ matrices with real values and we know $\det(AB-BA)=0$. Show that equation $\det(A+xB)=0$ has real solutions if and only if $$\det(A^{2}+B^{2})\geq(\det(A)+\det(B))^{2}.$$
I used the formula:
$$\det(A+xB)=\det(A)+x^{2}\det(B)+x(\det(A+B)-\det(A)-\det(B)).$$ To have real solutions for $x$, we need to have: $$(\det(A+B)-(\det(A)+\det(B)))^{2}\geq4\det(A)\det(B).$$
Now I dont know how to use the fact that $\det(AB-BA)=0$ Is there any formula to rewrite $\det(AB-BA)$ in terms of $\det(A+B)$, $\det(A)$, $\det(B)$?
 A: As noted in the comments, you need some additional assumptions for the statement to be true. I'll assume $B$ is invertible. Since these are $2\times 2$ matrices we could probably prove the statement fairly easily by just computing with elements, but I'm going to use some more advanced machinery at the risk of it being overkill.
For any real $2\times 2$ matrices $X$ and $Y$ we have
$$\det(X+Y)=\det(X)+\mathop{\mathrm{tr}}(X\mathbin{\square}Y)+\det(Y)\tag{1}$$
where $X\mathbin{\square}Y$ denotes the $1\times 1$ box product of $X$ and $Y$.
Note the formula you used for $\det(A+xB)$ can be obtained by first taking $X=A$ and $Y=xB$ and then taking $X=A$ and $Y=B$ in (1). Also, the inequality you obtained is equivalent to
$$\mathop{\mathrm{tr}}[(A\mathbin{\square}B)^2]=[\mathop{\mathrm{tr}}(A\mathbin{\square}B)]^2\ge 4\det(AB)\tag{2}$$
Taking $X=AB$ and $Y=-BA$ in (1) yields
$$0=\det(AB-BA)=2\det(AB)-\mathop{\mathrm{tr}}(AB\mathbin{\square}BA)$$
so
$$\mathop{\mathrm{tr}}(AB\mathbin{\square}BA)=2\det(AB)\tag{3}$$
Taking $X=A^2$ and $Y=B^2$ in (1) yields
$$\det(A^2+B^2)=\det(A^2)+\mathop{\mathrm{tr}}(A^2\mathbin{\square}B^2)+\det(B^2)$$
so the desired inequality is
$$\mathop{\mathrm{tr}}(A^2\mathbin{\square}B^2)\ge 2\det(AB)\tag{4}$$
By the Greub-Vanstone identity (see the link above),
$$(A\mathbin{\square}B)^2=A^2\mathbin{\square}B^2+AB\mathbin{\square}BA\tag{5}$$
Now (2) is equivalent to (4) by (3) and (5), which establishes the result (assuming $B$ invertible).
A: As I have mentioned in my comment under your question, the statement is false, e.g. when $A=I$ and $B=0$. It can be salvaged by assuming that $\det(B)\ne0$.
The other answer has probably explained where this question came from. Yet, being an Olympiad-typed question, I guess it was intended to be solved by using more familiar identities for real $2\times2$ matrices. Here I think we need the following two:

*

*$\det(X+Y)=\det(X)+\det(Y)+\operatorname{tr}\left(Y\operatorname{adj}(X)\right)$;

*$|\det(X+iY)|^2=\det(X^2+Y^2)-\det(XY-YX)$.

(While the quantity $\operatorname{tr}\left(Y\operatorname{adj}(X)\right)$ in the first identity does not appear symmetric at the outset, with entrywise computation one can easily verify that $\operatorname{tr}\left(X\operatorname{adj}(Y)\right)=\operatorname{tr}\left(Y\operatorname{adj}(X)\right)$.)
Now let $a=\det(A),b=\det(B)$ and $t=\operatorname{tr}\big(B\operatorname{adj}(A)\big)$. Using the two identities above with the condition that $\det(AB-BA)=0$, we obtain
$$
\begin{cases}
\det(A+xB)=bx^2+tx+a,\\
\det(A^2+B^2)= |\det(A+iB)|^2=|-b+it+a|^2=(a-b)^2+t^2.
\end{cases}
$$
So, essentially, you are asked to show that $bx^2+tx+a=0$ has a real root if and only if $(a-b)^2+t^2\ge(a+b)^2$. But this is true when $b\ne0$ because both propositions are equivalent to $t^2\ge4ab$.
