# Singular covariance matrix

I am looking into the process $\{X_t, t\in\mathbb{Z}\}$, $X_t=A\cos(\lambda t)+B\sin(\lambda t)$, here $\lambda\in(0,\pi)$ is fixed, $A$ and $B$ are uncorrelated random variables with $EA=EB=0$, $EA^2=EB^2=\sigma^2$.

I have found the covariance function $r(k)=\sigma^2\cos(\lambda k)$ and now I want to show that process' covariance matrix

$$\sigma^2 \begin{pmatrix} 1 & \cos(\lambda) & \cos(2\lambda) & \cdots & \cos(n\lambda) \\ \cos(\lambda) & 1 & \cos(\lambda) & \cdots & \cos((n-1)\lambda) \\ \cdots & \cdots & \cdots & \cdots & \cdots \\ \cos(n\lambda) & \cos((n-1)\lambda) & \cos((n-2)\lambda) & \cdots & 1 \end{pmatrix}$$

is singular when $n\geq 2$. Also the relationship $X_{n+1}=2X_n\cos\lambda-X_{n-1}$, $n\geq 2$ holds, from which I get that $r(k)=\frac{r(k-1)+r(k+1)}{2\cos\lambda}$. But I can not find a fast way to show that the covariance matrix is singular neither using latter relationship nor the matrix above.

-
Is $A$ and $B$ independent? – Patrick Li Jan 12 '13 at 3:45
@PatrickLi, uncorrelated, thank you for reminding. – Julius Jan 12 '13 at 3:47
The rank of the matrix isn't full, is it? – Stefan Hansen Jan 12 '13 at 9:27
@StefanHansen, it isn't, I have checked $3\times3$, $4\times4$ cases with $\lambda=1$ and the rank was 2. – Julius Jan 12 '13 at 10:27

Let $Z = (X_0 , X_1 , \dots , X_n )^T$,

and let $$U = \left( \begin{matrix}1 & 0 \\ \cos \lambda & \sin \lambda \\ \dots & \dots \\ \cos \lambda n &\sin \lambda n \end{matrix} \right).$$

Then $$Z = U \left( \begin{matrix} A \\ B \end{matrix} \right)$$ where $$\text{Cov} \left( \begin{matrix} A \\ B \end{matrix} \right ) = \left( \begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix} \right).$$ $EZ = 0$ and $$\text{Cov} Z = EZZ^T = U \text{Cov} \left( \begin{matrix} A \\ B \end{matrix} \right) U^T = U\left( \begin{matrix} 1 & 0 \\ 0 & 1\\ \end{matrix} \right) U^T = UU^T.$$

Clearly, the covariance matrix of $Z$ is singular for $n \geq 2$ as rank of $U$ is atmost 2 as it is a $n \times 2$ matrix and rank of $\text{Cov}(Z)$ is less or equal to the rank of $U$.

-
Could you explain what implies that the covariance is singular? – Julius Jan 13 '13 at 1:06
The rank of a product of matrices is less than or equal to the rank of any matrix in the product. – Arin Chaudhuri Jan 13 '13 at 10:17
Nice solution. This is something blindingly obvious that I didn't see. :-D – user1551 Jan 13 '13 at 10:18

The covariance matrix is nonsingular when $n\le2$ and $\lambda\in(0,\pi)$. It is singular for all real $\lambda$ when $n\ge3$.

Define $Y_1=X_1,\ Y_{2k}=X_{2k}$ and $Y_{2k+1}=X_{2k+1}+X_{2k-1}$ for $k=1,2,\ldots$. That is, $$\mathbf{Y}=\begin{pmatrix}Y_1\\Y_2\\Y_3\\ \vdots\end{pmatrix} =\begin{pmatrix} 1\\ 0&1\\ 1&0&1\\ &0&0&1\\ &&1&0&1\\ &&&\ddots&\ddots&\ddots \end{pmatrix} \begin{pmatrix}X_1\\X_2\\X_3\\ \vdots\end{pmatrix} =P\,\mathbf{X}\ \text{(say)}.$$ From the relationship $X_{n+1}=2X_n\cos\lambda-X_{n-1}$, we get $Y_{2k+1}=(2\cos\lambda) Y_{2k}$. Hence $E(Y_{2k+1}Y_i)=(2\cos\lambda)\,E(Y_{2k}Y_i)$, i.e. the $(2k+1)$-th row of the covariance matrix $E(\mathbf{Y}\mathbf{Y}^T)$ is a constant multiple of the $2k$-th row for every $k$. Hence $E(\mathbf{Y}\mathbf{Y}^T)$ is singular. Yet $E(\mathbf{Y}\mathbf{Y}^T)=P\,E(\mathbf{X}\mathbf{X}^T)\,P^T$. So $E(\mathbf{X}\mathbf{X}^T)$ is singular too. (Using the same argument, actually it can be show that the covariance matrix of $\mathbf{X}$ has rank 2.)

-

Hmm, if you imagine your described conditions in the multidimensional space, where the relations between vectors are described by the same multiple of an angle $\lambda$, such that
$\cos(X_0,X_1)=\cos(X_1,X_2)=\cos(X_k,X_{k+1})=\cos(\lambda)$
and the angles between vectors, whose index differs by 2, have the double value $2 \lambda$ such that
$\cos(X_0,X_2)=\cos(X_1,X_3)=\cos(X_k,X_{k+2})=\cos(2 \lambda)$,
and that the vectors, whose index differs by 3 , have the triple angle - then it seems obvious, that all that vectors must lay in one plane in that multidimensional space...

It might be better imaginable, if you do not look at the set of cosines but on the set of arc-cosines, that means the true values of the angles - then it is easy to see, that three vectors from the same origin with the following angles between them: $\operatorname{angle}(X_0,X_1)=\lambda$, $\operatorname{angle}(X_1,X_2)=\lambda$, $\operatorname{angle}(X_0,X_2)=2\lambda$, must all lay on a plane (and that and how this is extensible to more vectors with the same pattern): the dimensionality of the space spanned by the $X$-variables is 2 and because the rank of the covariance-matrix ist the dimensionality the rank of the covariance-matrix is also 2 .

-