# Consistency of matrix equations for diagonalization/eigendecomposition with degenerate eigenvalues

I have three equations that seem consistent at the abstract level, but when I implement them I am getting mistakes when there are degenerate eigenvalues. They are

$$A = Q D Q^T$$ $$B = (Q^{-1})^T D Q^{-1}$$ $$AB = Q D^2 Q^{-1}$$

where $A$, $B$, are real symmetric matrices. Notice that $Q$ is a the matrix whose columns are the eigenvectors of $AB$. On the surface these look okay. Clearly multiplying $A$ and $B$ gives $AB$.

Since $A$, $B$ are normal matrices, they should admit a decomposition $A = U \tilde{D} U^T$ for a orthogonal matrix $U$. $Q$ is not an orthogonal matrix. If it was, then that would imply that $AB$ is normal, which it is not in general. Equivalently, it would imply $A=B$ which again is not true in general.

I thought that if $A$ could be diagonalized via $Q D Q^T$, then it was unique up to permutation of diagonal entries and multiplication of the columns of $Q$. But it seems that $U \tilde{D} U^T$ would also diagonalize $A$. Can I relate $Q$ to $U$ somehow? $U$ is special since $U^{-1} = U^T$, but $Q^{-1} \neq Q^T$, how can they possibly be related?

How can these equations be consistent? And are there certain problems when where are degenerate eigenvalues?

Generally the relation $A = Q D Q^T$ with $Q$ being invertible but not necessarily orthogonal is known as matrix congruence. A symmetric matrix is always congruent to a diagonal matrix, but note that the diagonal matrix $D$ does not (necessarily) consist of eigenvalues of $A$ in this case, but by Sylvester's law of inertia $D$ has the same number of positive sign, negative sign and zero-valued eigenvalues as $A$.
In this answer you can see a typical algorithm to find the matrices $D$ and $Q$ if you have $A$.
So just to sum up, the situation you have there: $Q$ diagonalizes $AB$ in the usual similarity sense, and $D^2$ contains the eigenvalues of $AB$. But the other two equations do not indicate diagonalization in the eigendecomposition sense, it just shows that $A$ and $B$ is congruent to $D$. Since congruence is also an equivalence relation this means $A$ and $B$ are congruent. All this allows us to say, is that $A$ and $B$ have the same number of positive, negative and zero-valued eigenvalues, no more...