# Peculiar Behaviour of Conditional Variance for Multivariate Normal Distributions

Let $Z_i \sim \mathcal{N}(0,1)$ be independent normal distributions. Consider the following correlated variables, defined by $$X_1 = \frac{Z_1 + Z_2}{\sqrt{2}},\;\;\;X_2= \frac{Z_2 + Z_3}{\sqrt{2}},\;\;\;X_3= \frac{Z_3 + Z_4}{\sqrt{2}},\ldots$$

Thus each $X_i$ by itself is also a standard normal distribution but is correlated to the immediate neighbours $X_{i-1}$ and $X_{i+1}$. Consider the joint distribution of $(X_1,X_2)$ which is a joint normal with mean = $(0,0)$ and covariance matrix $$\begin{pmatrix} 1 & 1/2 \\ 1/2 & 1 \end{pmatrix}$$

Now the thing is according to the rules of conditional probability the conditional variance for $X_1$ is $\left(1-\rho^2\right)\sigma_1^2 = \frac{3}{4}$ in this case. So far so good.

Suppose we then consider the joint normal $\left(X_1,X_2,X_3\right)$, which has the covariance matrix $$\begin{pmatrix} 1 & 1/2 & 0 \\ 1/2 & 1 & 1/2 \\ 0 & 1/2 & 1 \end{pmatrix}$$

In this case, the conditional variance of $X_1$ is given by

$$1 - \begin{pmatrix} 1/2 & 0 \end{pmatrix}\begin{pmatrix} 1 & 1/2 \\ 1/2 & 1 \end{pmatrix}^{-1}\begin{pmatrix} 1/2 \\ 0 \end{pmatrix} = 2/3$$

The questions I have now are:

1. Since $X_1$ is not dependent on $X_3$ at all, why the conditional variance of $X_1$ drops from $3/4$ to $2/3$ when $X_3$ is taken into account?
2. If I further include $X_4,X_5,\ldots$ the conditional variance seems to drop further and reaches a limit of $1/2$ when I include a very large number of $X_i$. Is there any intuitive explanation for this limit?
• Re 1, note that ,in general, the conditional distributions of $X$ conditionally on $Y$ and conditionally on $(Y,Z)$ coincide when $Z$ is independent of $(X,Y)$ and that simply the independence that $Z$ and $X$ does not suffice. Roughly speaking, in your case, $X_3$ does bring some information on $X_1$ through $X_2$. Re 2, consider the model $$X_i=\frac{Z_i+aZ_{i+1}}b$$ with $b^2=1+a^2$ and $a<1$, then $$Z_i=bX_i-aZ_{i+1}$$ for every $i$ hence $$bX_1=Z_1+aZ_2=Z_1+abX_2-a^2Z_3=Z_1+abX_2-a^2bX_3+a^3bZ_4=Z_1+bY_2$$ with $$Y_2=\sum_{n=2}^\infty (-1)^na^{n-1}X_n$$ since $a^nbX_{n+1}\to0$ almost ... – Did Oct 6 '16 at 11:23
• ... surely. Thus, conditionally on $(X_n)_{n\geqslant2}$, $X_1$ is normal with mean $Y_2$ and variance $\frac1{b^2}$. When $a\to1$, one recovers your model and $b^2\to2$ -- hence the intuitive explanation of the limiting conditional variance $\frac12$ you observed. – Did Oct 6 '16 at 11:24