# Finding Orthonormal Eigenvectors of a $4x4$ Variance-Covariance Matrix

Let $$U_1$$ and $$U_2$$ be two independent uniform random variables on $$[0,1]$$. Suppose that $$X= (X_1,X_2,X_3,X_4)′$$where $$X_1=U_1$$, $$X_2=U_2$$, $$X_3=U_1+U_2$$ and $$X_4=U_1−U_2$$.

(a) Compute the variance covariance matrix $$\Sigma$$ of $$X$$.

(b) Verify that $$v_1=\frac{1}{\sqrt{3}}(0,1,1,−1)$$, $$v_2=\frac{1}{\sqrt{3}}(1,0,1,1)$$, $$v_3=\frac{1}{\sqrt{6}}(0,2,−1,1)$$, and $$v_4=\frac{1}{\sqrt{6}}(2,0,−1,−1)$$ are orthonormal eigenvectors of $$\Sigma$$. Find out the corresponding eigenvalues.

We have that $$\Sigma=\begin{bmatrix}\frac{1}{12}&0&\frac{1}{12}&\frac{1}{12}\\0&\frac{1}{12}&\frac{1}{12}&-\frac{1}{12}\\ \frac{1}{12}&\frac{1}{12}&\frac{1}{6}&0\\ \frac{1}{12}&-\frac{1}{12}&0&\frac{1}{6}\end{bmatrix}$$

since $$\mathsf{Var}(X_1)=\mathsf{Var}(X_2)=\mathsf{Var}(U_1)=\frac{1}{12}$$, $$\mathsf{Var}(X_3)=\mathsf{Var}(U_1+U_2)=\frac{1}{6}$$, $$\mathsf{Cov}(X_1,X_2)=0$$, $$\mathsf{Cov}(X_1,X_3)=\mathsf{Cov}(U_1,U_1+U_2)=\mathsf{Cov}(U_1,U_1)+\mathsf{Cov}(U_1,U_2)=\mathsf{Var}(U_1)=\frac{1}{12}$$, $$\mathsf{Cov}(X_1,X_4)=\mathsf{Cov}(U_1,U_1-U_2)=\mathsf{Cov}(U_1,U_1)-\mathsf{Cov}(U_1,U_2)=\mathsf{Var}(U_1)=\frac{1}{12}$$, $$\mathsf{Cov}(X_2,X_3)=\mathsf{Cov}(U_2,U_1+U_2)=\mathsf{Cov}(U_1,U_2)+\mathsf{Cov}(U_2,U_2)=\mathsf{Var}(U_2)=\frac{1}{12}$$, $$\mathsf{Cov}(X_2,X_4)=\mathsf{Cov}(U_2,U_1-U_2)=\mathsf{Cov}(U_1,U_2)-\mathsf{Cov}(U_2,U_2)=-\mathsf{Var}(U_2)=-\frac{1}{12}$$, and $$\mathsf{Cov}(X_3,X_4)=\mathsf{Cov}(U_1+U_2,U_1-U_2)=\mathsf{Cov}(U_1,U_1)-\mathsf{Cov}(U_1,U_2)+\mathsf{Cov}(U_1,U_2)-\mathsf{Cov}(U_2,U_2)=\frac{1}{12}-\frac{1}{12}=0$$

Software then gives that

\begin{align*} det(\Sigma-\lambda I) &=det\left(\begin{bmatrix}\frac{1}{12}-\lambda&0&\frac{1}{12}&\frac{1}{12}\\0&\frac{1}{12}-\lambda&\frac{1}{12}&-\frac{1}{12}\\ \frac{1}{12}&\frac{1}{12}&\frac{1}{6}-\lambda&0\\ \frac{1}{12}&-\frac{1}{12}&0&\frac{1}{6}-\lambda\end{bmatrix}\right)\\\\ &=det\left(\begin{pmatrix}\frac{1}{12}-\lambda&0&\frac{1}{12}&\frac{1}{12}\\ 0&\frac{1}{12}-\lambda&\frac{1}{12}&-\frac{1}{12}\\ 0&0&\frac{3\lambda\left(4\lambda-1\right)}{1-12\lambda}&0\\ 0&0&0&\frac{3\lambda\left(4\lambda-1\right)}{1-12\lambda}\end{pmatrix}\right)\\\\ &=\frac{\lambda^2\left(1-4\lambda\right)^2}{16} \end{align*}

Hence $$\lambda_1=\frac{1}{4}$$ and $$\lambda_2=0$$

We have that for $$\lambda_2=0$$,

$$\begin{bmatrix}\frac{1}{12}&0&\frac{1}{12}&\frac{1}{12}\\0&\frac{1}{12}&\frac{1}{12}&-\frac{1}{12}\\ \frac{1}{12}&\frac{1}{12}&\frac{1}{6}&0\\ \frac{1}{12}&-\frac{1}{12}&0&\frac{1}{6}\end{bmatrix}\begin{bmatrix}x_1\\x_2\\x_3\\x_4\end{bmatrix}=\begin{bmatrix}0\\0\\0\\0\end{bmatrix}$$

means $$x_2=-x_3+x_4$$ and $$x_1=-x_3-x_4$$. Letting $$x_3=x_4=1$$ we have that $$x_2=0$$ and $$x_1=-2$$

Hence the associated eigenvector is $$\frac{1}{\sqrt{6}}(-2,0,1,1)$$ which is equivalent to $$v_4$$ which was provided.

Similarly, we find that for $$\lambda_1=\frac{1}{4}$$ we get an associated eigenvector of $$\frac{1}{\sqrt{3}}(1,0,1,1)$$ which is $$v_2$$.

Where do $$v_1$$ and $$v_3$$ come from?

• You really don’t need to go through all of that work. You’ve already been given a set of purported eigenvectors. Simply verify for each one that $\Sigma v=\lambda v$ for some $\lambda$. – amd Apr 1 '19 at 3:14
• As far as the specific question you’re asking goes, each of the eigenspaces is two-dimensional. You’ve only computed a single eigenvector in each. – amd Apr 1 '19 at 3:15
• For your second point, are you saying that I should be looking for another set of solutions for $\lambda=0$ and $\lambda=\frac{1}{4}$? – Remy Apr 1 '19 at 3:24
• I think I understand now so I'll just answer my own question. Just to make sure, for an orthonormal eigenenvector does it hold that $v_1=-v_1$? – Remy Apr 1 '19 at 3:31
• Any nonzero scalar multiple of an eigenvector is also an eigenvector with the same eigenvalue. – amd Apr 1 '19 at 6:09

Although finding eigenvectors and eigenvalues of this covariance matrix for yourself is good practice, it’s not really necessary to do so here since you’ve been given a set of vectors and asked to verify that they are indeed eigenvectors of the matrix. This is a straightforward matter of multiplying each one by the covariance matrix. For example, $$\Sigma \begin{bmatrix}0\\1\\1\\-1\end{bmatrix} = \begin{bmatrix}0\\\frac14\\\frac14\\-\frac14\end{bmatrix},$$ so $$(0,1,1,-1)^T$$ is an eigenvector with eigenvalue $$\frac14$$. As any nonzero scalar multiple of an eigenvector is also an eigenvector, this verifies that $$v_1$$ is an eigenvector of $$\Sigma$$ with eigenvalue $$\frac14$$. Each of the given vectors obviously has unit length, and it’s fairly easy to verify by inspection that they are pairwise orthogonal, so they do indeed form an orthonormal set.
As for your question about finding another pair of eigenvectors, note that both eigenvalues that you computed from the characteristic equation have algebraic multiplicity $$2$$. Every real symmetric matrix is diagonalizable, so their geometric multiplicity is also $$2$$, i.e., the corresponding eigenspaces are two-dimensional. Using the usual method of Gaussian elimination, $$\Sigma$$ reduces to $$\begin{bmatrix}1&0&1&1\\0&1&1&-1\\0&0&0&0\\0&0&0&0\end{bmatrix},$$ from which we can read that the eigenspace of $$0$$ is spanned by $$w_1=(1,1,-1,0)^T$$ and $$w_2=(1,-1,0,-1)^T$$. Every linear combination of these two vectors is an eigenvector of $$0$$. In particular, $$v_3=\frac1{\sqrt6}(w_1-w_2)$$ and $$v_4=\frac1{\sqrt6}(w_1+w_2)$$. Similarly, $$\Sigma-\frac14I$$ reduces to $$\begin{bmatrix}1&0&-\frac12&-\frac12\\0&1&-\frac12&\frac12\\0&0&0&0\\0&0&0&0\end{bmatrix},$$ so the eigenspace of $$\frac14$$ is spanned by $$(1,1,2,0)^T$$ and $$(1,-1,0,2)^T$$. I’ll leave finding the linear combinations of these two vectors that produce $$v_1$$ and $$v_2$$ to you.
For $$\lambda=0$$, we could also let $$x_3=-1$$ and $$x_4=1$$. Hence $$x_2=2$$ and $$x_1=0$$ so we get another eigenvector of $$\frac{1}{\sqrt{6}}(0,2,-1,1)$$
For $$\lambda=\frac{1}{4}$$, we could also let $$x_3=1$$ and $$x_4=-1$$. Hence $$x_2=1$$ and $$x_1=0$$ so we get another eigenvector of $$\frac{1}{\sqrt{3}}(0,1,1,-1)$$