# Eigenvalues of a Covariance Matrix with Noise

Imagine to have a covariance matrix $$2\times 2$$ called $$\Sigma^*$$.

$$\begin{bmatrix}1+\sigma^2&\rho_{12}\\\rho_{21}&1+\sigma^2\end{bmatrix}$$

I know that $$\rho_{12} = \rho_{21}$$ because it's symmetric.

I proceed in calculating the eigenvalues from

$$\det(\Sigma^*-\lambda I) = \lambda^2 - 2\lambda(1+\sigma^2) + (1+\sigma^4+2\sigma^2-\rho_{12}^2).$$

and finally the eigenvalues are:

$$\lambda_1 = 1+\sigma^2 + \rho_{12}$$

$$\lambda_2 = 1+\sigma^2 - \rho_{12}.$$

Now I've tried to do the same with a $$3 \times 3$$ matrix

$$\Sigma^* = \begin{bmatrix}1+\sigma^2&\rho_{12}&\rho_{13}\\\rho_{21}&1+\sigma^2&\rho_{23}\\\rho_{31}&\rho_{32}&1+\sigma^2\end{bmatrix}.$$

I know that $$\rho_{12} = \rho_{21}, \rho_{13} = \rho_{31}, \rho_{23} = \rho_{32}$$ because it's symmetric.

Now I'm kinda stuck. I'll leave below what I did:

$$\det(Σ^*-\lambda I) = (1+\sigma^2 -\lambda)^3 - (1+\sigma^2 -\lambda)(\rho_{12}^2 + \rho_{13}^2 + \rho_{23}^2) + 2 \rho_{12}\rho_{13}\rho_{23}$$

and if $$z = 1+\sigma^2 -\lambda$$

$$\det(Σ^*-\lambda I) = z^3 - z(\rho_{12}^2 + \rho_{13}^2 + \rho_{23}^2) + 2 \rho_{12}\rho_{13}\rho_{23}.$$

How can I go further? Is it better to either find the values of z or to use another approach like SVD if applicable?

My main goal is to find those eigenvalues.

****** UPDATE ******

Following the answer below from @GNUSupporter 8964民主女神 地下教會, I see that since the discriminant is less or equal to zero, I have that

$$u^3, v^3 = -\dfrac{q}{2} ± i\sqrt{-\Delta}$$

their module is $$R = \sqrt{(-\frac{q}{2})^2+(\sqrt{-\Delta})^2}$$

Do I need $$R$$? If yes, I need to calculate $$\theta = arctg (-\frac{\sqrt{-\Delta}}{-\frac{q}{2}})$$

If not, is it possible to just say that

$$x = u + v = \sqrt[3]{-\dfrac{q}{2} + i\sqrt{-\Delta}} + \sqrt[3]{-\dfrac{q}{2} - i\sqrt{-\Delta}}?$$

• Real symmetric matrices are always diagonalisable. – GNUSupporter 8964民主女神 地下教會 Apr 30 '19 at 13:44
• @GNUSupporter8964民主女神地下教會 uhm, ok. So, how is this answer related to my question? – JackLametta Apr 30 '19 at 13:46
• Therefore, the characteristic polynomial of the covariance matrix is always solvable in $\Bbb{R}$, so you don't need SVD. – GNUSupporter 8964民主女神 地下教會 Apr 30 '19 at 13:50
• @GNUSupporter8964民主女神地下教會 so, could you provide me support in resolving that equation to find its roots? – JackLametta Apr 30 '19 at 13:50
• I've posted a solution in response to your comment. Please check. – GNUSupporter 8964民主女神 地下教會 Apr 30 '19 at 14:31

My main goal is to find those eigenvalues.

To find $$\Sigma^*$$'s eigenvalues, apply Cardano's method with $$p = -(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2)$$ and $$q = 2 \rho_{12}\rho_{23}\rho_{31}$$ to your last equality

$$\det(Σ^*-\lambda I) = z^3 - z(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2) + 2 \rho_{12}\rho_{23}\rho_{31}. \tag1 \label1$$

We introduce variables $$u$$ and $$v$$ so that $$z = u + v$$

\begin{align} u^{3} &= -{q \over 2}+{\sqrt {{q^{2} \over 4}+{p^{3} \over 27}}} = -\rho_{12}\rho_{23}\rho_{31} + \sqrt{(\rho_{12}\rho_{23}\rho_{31})^2 - \frac{(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2)^3}{27}} \tag2 \label2 \\ v^{3} &= -{q \over 2}-{\sqrt {{q^{2} \over 4}+{p^{3} \over 27}}} = -\rho_{12}\rho_{23}\rho_{31} - \sqrt{(\rho_{12}\rho_{23}\rho_{31})^2 - \frac{(\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2)^3}{27}}. \tag3 \label3 \end{align}

Note that by the ,

$$\sqrt[3]{\rho_{12}^2 \rho_{23}^2 \rho_{31}^2} \le \frac{\rho_{12}^2 + \rho_{23}^2 + \rho_{31}^2}{3},$$

so the discriminant $$\Delta = {q^{2} \over 4}+{p^{3} \over 27} \le 0.$$

$$\Delta = 0$$ iff equality holds in the AM-GM inequality iff $$\rho_{12}^2 = \rho_{23}^2 = \rho_{31}^2$$.

Observe from \eqref{2} and \eqref{3} that $$u^3 = \overline{v^3}$$. Denote $$\omega$$ the third root of unity not equal to $$1$$. (i.e. $$\omega^3 = 1$$ but $$\omega \ne 1$$.)

Since all eigenvalues of a real symmetric matrix are real, you just take $$u + \overline{u}$$, $$\omega u + \overline{\omega u}$$ and $$\omega^2 u + \overline{\omega^2 u}$$ as roots for \eqref{1}, where $$u$$ is fixed as any one of the three roots of \eqref{2}.

It doesn't matter which root of \eqref{2} is chosen since $$\omega$$ permutes the three roots, so eventually, all three roots of \eqref{2} are covered.

To finish this question, just do the substitution $$\lambda = 1+\sigma^2 -z$$.

Edit in response to OP's comment

Despite OP's edit to include some trigonometric function, in this section, I shall make no use of them since this is essentially an algebra problem for finding of , and that trigonometric functions are transcendental. I'll keep my existing working algebraic at the first place. By algebra, I don't mean to be arithmetic, that is, I avoid as much calculations as possible.

Each of equations \eqref{2} and \eqref{3} have three distinct roots, so there are at most $$3 \times 3 = 9$$ candidates for $$z = u + v$$.

I say "at most" since I haven't (and won't directly) checked some of them repeat themselves.

The reason for \eqref{2} and \eqref{3} to have three distinct roots is that the cubic equation $$\omega^3 = 1$$ has three distinct roots: $$1$$ and the two conjugate complex roots $$\omega$$ and $$\bar{\omega} = \omega ^2$$, so that $$\omega^2 + \omega + 1 = 0$$. If you fix a root $$u$$, then $$\omega u$$ and $$\omega^2 u$$ are also roots to \eqref{2} since $$(\omega u)^3 = \omega^3 u^3 = u^3$$ and $$(\omega^2 u)^3 = (\omega^3)^2 u^3 = u^3$$. Since we want to represent all three roots of \eqref{2} as $$u, \omega u, \omega^2 u$$, we don't want $$\omega = 1$$.

To visualize $$u$$, you may consider it as a nonzero arrow on the complex plane, and $$\omega$$ as a rotation of 120°.

Third roots of unity
Image source: Wikimedia commons

Given that \eqref{2} is solved and that $$u^3 = \overline{v^3}$$, the easiest root for \eqref{3} would be $$v = \bar{u}$$. Apply the same argument above to see that roots of \eqref{3} are $$\bar{u}, \omega \bar{u}, \omega^2 \bar{u}$$.

The remaining work is too select the three real values from $$\omega^i u + \omega^j \bar{u}$$, $$i,j \in \{0,1,2\}$$. The easiest choice is the case when $$i = j = 0$$, so that $$z = u + \bar{u} \in \Bbb{R}$$. It should be quite obvious that we should choose $$\omega u + \overline{\omega u}$$ and $$\omega^2 u + \overline{\omega^2 u}$$. These three roots are distinct since rotation $$\omega \ne 1$$. Since \eqref{1} is at most three roots, we're done.

To conclude, to solve for $$z$$, fix any root $$u$$ in \eqref{2}, then the first root is $$2\mathrm{Re}(u)$$. Rotate $$u$$ by $$120°$$ (anti-)clockwisely to get another root $$2\mathrm{Re}(\omega u)$$, repeat this to get the last root $$2\mathrm{Re}(\omega^2 u)$$.

Remark: I used "third root of unity" in the first version of this answer since it's easier to type in words than its explicit expression.

Edit in response to updated question body

Do I need $$R$$?

As shown by the previous section, we can solve the problem without introducing the modulus $$R = \sqrt{(-\frac{q}{2})^2+(\sqrt{-\Delta})^2}$$ and $$\theta = \arctan\frac{-\sqrt{-\Delta}}{-q/2}$$ of $$u^3$$ (or $$v^3$$).

Nonetheless, $$R$$ and $$\theta$$ provide a more compact expression for $$u^3$$ and $$v^3$$: $$v^3 = Re^{i\theta}$$ and $$u^3 = Re^{-i\theta}$$. Therefore, $$u = \sqrt[3]{R} e^{-i(\theta + 2k\pi)/3}$$ and $$v = \sqrt[3]{R} e^{i(\theta + 2m\pi)/3}$$, $$k,m \in \{0,1,2\}$$. The checking for $$u + v \in \Bbb{R}$$ is similar to the previous section, and I left this as exercise.

To litterally respond to your question, it depends on your personal choice. Nobody can answer that for you.

• If you don't wish to overload some transcendental stuff like $$\arctan$$, $$e$$, Euler's identity, then the answer is "no".
• Otherwise, it's "yes".

If not, is it possible to just say that $$x = u + v = \sqrt[3]{-\dfrac{q}{2} + i\sqrt{-\Delta}} + \sqrt[3]{-\dfrac{q}{2} - i\sqrt{-\Delta}}?$$

First, I suppose you mean $$z = u + v$$, since $$x$$ has never been used before. Second, this notation for cubic root $$\sqrt[3]{\cdots}$$ is ambiguous unless $$\Delta = 0$$. The notation $$\sqrt[3]{\cdots}$$ can mean any one of the tree cubic root unless the number inside the radical sign is real.

To see this ambiguity, consider $$\sqrt[3]{i}$$. One graphical way to interpret this is to trisect a right angle: $$3 \times \frac\pi6 = \frac\pi2$$. However, there are two more possible visualisations: $$3 \times \frac{5\pi}{6} = \frac{5\pi}{2}$$ and $$3 \times \frac{3\pi}{2} = \frac{9\pi}{2}$$. Therefore, the equation $$z^3 = i$$ has three distinct roots $$e^{\pi/6}$$, $$e^{5\pi/6}$$ and $$e^{3\pi/2}$$. One way to decide which root $$\sqrt[3]{i}$$ should take is the choose the least nonnegative argument. This is called the principal root. Wolfram Alpha's calculation for $$\sqrt[3]{i}$$ adopts this definition, but that's not universally accepted according to Wikipedia's page on $$n$$-th root (search for principal root on the page to see this). This can be explained by the observation that conjugation is not preserved by the action of taking principal root: $$i$$ and $$-i$$ are conjugate to each other, but their principal roots ($$e^{\pi/6}$$ and $$e^{\pi/2} = i$$) aren't.

This ambiguity would give rise to at most $$3 \times 3 = 9$$ possible candidates of $$z$$, but a cubic equation admits exactly three roots in $$\Bbb{C}$$. Therefore, an additional condition has to be added.

$$uv = -\dfrac{p}{3}$$

This condition comes from Cardano's trick $$z = u + v$$, which transforms the depressed cubic equation $$z^3 + pz + q = 0$$ into \begin{aligned} (u+v)^3 + p(u+v) + q &= u^3 + v^3 + 3uv(u+v) + p(u+v) + q \\ &= u^3 + v^3 + (3uv + p)(u+v) + q = 0 \end{aligned}

Set $$3uv + p = 0$$. Since a new variable is introduced, we can add one more constraint. This is where the $$uv = -\frac{p}{3}$$ from. Then the above equation becomes $$u^3 + v^3 = -q$$. To solve for $$u$$ and $$v$$, we make use of the well-known quadratic formula on $$u^3$$ and $$v^3$$ to get \eqref{2} and \eqref{3}.

• Shouldn't the $\theta$ be calculated too? – JackLametta Apr 30 '19 at 17:00
• What do you mean by $\theta$? – GNUSupporter 8964民主女神 地下教會 Apr 30 '19 at 17:02
• I did not get from this part on: so 𝑢3=𝑣3 Denote 𝜔 the third root of unity. (i.e. 𝜔3=1 but 𝜔≠1) Since all eigenvalues of a real symmetric matrix are real, you just take 𝑢+𝑢⎯⎯⎯ , 𝜔𝑢+𝜔𝑢⎯⎯⎯⎯⎯⎯⎯ and 𝜔2𝑢+𝜔2𝑢⎯⎯⎯⎯⎯⎯⎯⎯⎯ as roots for (1). – JackLametta Apr 30 '19 at 20:54
• I meant to say z and not x. Since I need eigenvalues to go ahead with other proofs, why the sum of u and its conjugates are not well posed in the quesiton? – JackLametta May 1 '19 at 8:36
• I mean the notation $\sqrt[n]{\cdots}$ is not well-defined for non-real numbers. It's hard to choose which one from the $n$ possible candidates. The cause for such difficulty is because $\Bbb{C}$ is not totally ordered like $\Bbb{R}$. That is, you can't say "$z < w$ in $\Bbb{C}$ as in $\Bbb{R}$". One possible choice (known as principal $n$-th root) from the $n$ roots is the one with minimum argument, but this is not universally accepted. I'm moving to chat to avoid an overflow of comments here. – GNUSupporter 8964民主女神 地下教會 May 1 '19 at 8:53