Singular covariance matrix

Question

I am looking into the process $\{X_t, t\in\mathbb{Z}\}$, $X_t=A\cos(\lambda t)+B\sin(\lambda t)$, here $\lambda\in(0,\pi)$ is fixed, $A$ and $B$ are uncorrelated random variables with $EA=EB=0$, $EA^2=EB^2=\sigma^2$.

I have found the covariance function $r(k)=\sigma^2\cos(\lambda k)$ and now I want to show that process' covariance matrix

$$\sigma^2 \begin{pmatrix} 1 & \cos(\lambda) & \cos(2\lambda) & \cdots & \cos(n\lambda) \\ \cos(\lambda) & 1 & \cos(\lambda) & \cdots & \cos((n-1)\lambda) \\ \cdots & \cdots & \cdots & \cdots & \cdots \\ \cos(n\lambda) & \cos((n-1)\lambda) & \cos((n-2)\lambda) & \cdots & 1 \end{pmatrix} $$

is singular when $n\geq 2$. Also the relationship $X_{n+1}=2X_n\cos\lambda-X_{n-1}$, $n\geq 2$ holds, from which I get that $r(k)=\frac{r(k-1)+r(k+1)}{2\cos\lambda}$. But I can not find a fast way to show that the covariance matrix is singular neither using latter relationship nor the matrix above.

Answer 1

Let $Z = (X_0 , X_1 , \dots , X_n )^T$,

and let $$U = \left( \begin{matrix}1 & 0 \\ \cos \lambda & \sin \lambda \\ \dots & \dots \\ \cos \lambda n &\sin \lambda n \end{matrix} \right).$$

Then $$ Z = U \left( \begin{matrix} A \\ B \end{matrix} \right)$$ where $$\text{Cov} \left( \begin{matrix} A \\ B \end{matrix} \right ) = \left( \begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix} \right).$$ $EZ = 0$ and $$ \text{Cov} Z = EZZ^T = U \text{Cov} \left( \begin{matrix} A \\ B \end{matrix} \right) U^T = U\left( \begin{matrix} 1 & 0 \\ 0 & 1\\ \end{matrix} \right) U^T = UU^T.$$

Clearly, the covariance matrix of $Z$ is singular for $ n \geq 2 $ as rank of $U$ is atmost 2 as it is a $ n \times 2 $ matrix and rank of $ \text{Cov}(Z) $ is less or equal to the rank of $U$.

Answer 2

The covariance matrix is nonsingular when $n\le2$ and $\lambda\in(0,\pi)$. It is singular for all real $\lambda$ when $n\ge3$.

Define $Y_1=X_1,\ Y_{2k}=X_{2k}$ and $Y_{2k+1}=X_{2k+1}+X_{2k-1}$ for $k=1,2,\ldots$. That is, $$ \mathbf{Y}=\begin{pmatrix}Y_1\\Y_2\\Y_3\\ \vdots\end{pmatrix} =\begin{pmatrix} 1\\ 0&1\\ 1&0&1\\ &0&0&1\\ &&1&0&1\\ &&&\ddots&\ddots&\ddots \end{pmatrix} \begin{pmatrix}X_1\\X_2\\X_3\\ \vdots\end{pmatrix} =P\,\mathbf{X}\ \text{(say)}. $$ From the relationship $X_{n+1}=2X_n\cos\lambda-X_{n-1}$, we get $Y_{2k+1}=(2\cos\lambda) Y_{2k}$. Hence $E(Y_{2k+1}Y_i)=(2\cos\lambda)\,E(Y_{2k}Y_i)$, i.e. the $(2k+1)$-th row of the covariance matrix $E(\mathbf{Y}\mathbf{Y}^T)$ is a constant multiple of the $2k$-th row for every $k$. Hence $E(\mathbf{Y}\mathbf{Y}^T)$ is singular. Yet $E(\mathbf{Y}\mathbf{Y}^T)=P\,E(\mathbf{X}\mathbf{X}^T)\,P^T$. So $E(\mathbf{X}\mathbf{X}^T)$ is singular too. (Using the same argument, actually it can be show that the covariance matrix of $\mathbf{X}$ has rank 2.)

Answer 3

Hmm, if you imagine your described conditions in the multidimensional space, where the relations between vectors are described by the same multiple of an angle $\lambda$, such that
$\cos(X_0,X_1)=\cos(X_1,X_2)=\cos(X_k,X_{k+1})=\cos(\lambda)$
and the angles between vectors, whose index differs by 2, have the double value $2 \lambda$ such that
$\cos(X_0,X_2)=\cos(X_1,X_3)=\cos(X_k,X_{k+2})=\cos(2 \lambda)$,
and that the vectors, whose index differs by 3 , have the triple angle - then it seems obvious, that all that vectors must lay in one plane in that multidimensional space...

It might be better imaginable, if you do not look at the set of cosines but on the set of arc-cosines, that means the true values of the angles - then it is easy to see, that three vectors from the same origin with the following angles between them: $\operatorname{angle}(X_0,X_1)=\lambda$, $\operatorname{angle}(X_1,X_2)=\lambda$, $\operatorname{angle}(X_0,X_2)=2\lambda$, must all lay on a plane (and that and how this is extensible to more vectors with the same pattern): the dimensionality of the space spanned by the $X$-variables is 2 and because the rank of the covariance-matrix ist the dimensionality the rank of the covariance-matrix is also 2 .