How prove this two symmetric matrices $AB=0$ Let $A,B$ be real symmetric matrices, and for any $n\in \Bbb N^{+}$, and for all $x,y\in \Bbb R$, we have
$$tr(xA+yB)^n=x^ntr(A^n)+y^ntr(B^n).$$
Show that $AB=0$.
My try:since
$$(xA+yB)^n=x^nA^n+\binom{n}{1}x^{n-1}A^{n-1}yB+\cdots+\binom{n}{n-1}xA(yB)^{n-1}+y^nB^n$$
so
$$tr\left(x^nA^n+\binom{n}{1}x^{n-1}A^{n-1}yB+\cdots+\binom{n}{n-1}xA(yB)^{n-1}+y^nB^n\right)=x^ntr(A^n)+y^ntr(B^n)$$
then 
$$tr\left(\binom{n}{1}x^{n-1}A^{n-1}yB+\cdots+\binom{n}{n-1}xA(yB)^{n-1}\right)=0$$
so
$$\binom{n}{1}x^{n-1}ytr(A^{n-1}B)+\cdots+\binom{n}{n-1}xy^{n-1}tr(AB^{n-1})=0$$
then I can't
I found this is a nice result, maybe this is an odd problem, and I can't solve it. Thank you.
 A: This answer is an amalgamation of ideas from two deleted posts by @julien and @user1551 together with some of my thoughts.
Following @julien's insight, let us look at the polynomial condition at $n = 4$. 
If one compare the coefficient of the $x^2 y^2$ terms on both sides, we get
$$2\text{tr}(A^2B^2) + \text{tr}( (AB)^2 ) = 0$$
Combine this with the fact
$$\text{tr}((AB+BA)^2) = \text{tr}((AB+BA)^T(AB+BA)) \ge 0 \\\implies \text{tr}(A^2B^2) + \text{tr}( (AB)^2 ) \ge 0$$
We get $\text{tr}(A^2B^2) \le 0$. However
$$\text{tr}(A^2B^2) = \text{tr}(ABBA) = \text{tr}((BA)^\top BA ) \ge 0$$
This implies $\text{tr}(A^2B^2) = 0$.
Following @user1551's idea. Since $A$ is real symmetric, we can choose a basis such that 
$A$ is a diagonal matrix and its first $k=\text{rank}(A)$ diagonal entries are the only non-zero entries. $\text{tr}(A^2B^2)$ then become a positive linear combination of
the first $k$ diagonal entries of $B^2 = B^\top B$ which are non-negative themselves.
As a result, $\text{tr}(A^2 B^2) = 0$ implies the first k diagonal entries of $B^2$ vanish.
This means the first $k$ rows/columns of $B$ are zero and hence $AB=0$.
Update
It turns out there is a much simpler argument. The expression
$ \displaystyle \| C \|_F = \sqrt{ \text{tr}( C^\top C ) }$ 
is the famous Frobenius norm over the ring of real matrices! The vanishing of the coefficient of the $x^2 y^2$ terms 
$$\text{tr}((AB+BA)^2) + 2\text{tr}(BAAB) = 0$$
is equivalent to
$$\quad  \|AB+BA\|_F^2 + 2\|AB\|_F^2 = 0$$
This implies $\|AB\|_F = 0$ and hence $AB = 0$.
A: $(xA+yB)^n=x^nA^n+\binom{n}{1}x^{n-1}A^{n-1}yB+\cdots+\binom{n}{n-1}xA(yB)^{n-1}+y^nB^n$
I don't understand, this is not true in general, but only if $AB=BA$ ! But this is not supposed, isn't it ?...
I know a more powerful result : it is enough in fact that the equality holds for $x=1$ and $y=1$ to ensure that $AB=0$. Therafter we have immediately $BA=0$ exchanging the role of $A$ and $B$. 
And the converse is true, if $AB=0$ then $BA$=0 because of the symmetry (transpose) hence we have the equality of traces ... for all $x$ and $y$.
A: [Ignore this: Pick $x = 1, y = 0, n = 2$ to get $tr(A^2) = (tr A)^2$; you can do the same for $B$. {I've left it in only so that the comments still make sense.}]
Now pick $x = y = 1, n = 2$ to get $tr(A+B)^2 = tr(A^2) + tr(B^2)$. 
But $tr( (A+B)^2 ) = tr(A^2) + tr(AB + BA) + tr(B^2)$. 
So $tr(AB + BA) = 0$. So $2 tr(AB) = 0$. 
That should get you well on your way.  
