How to find the conditional distribution for multivariate normal RVs?

Question

Let $(X_1,X_2,X_3)$ be normally distributed with

\begin{align} \mu = \begin{pmatrix} \mu_1 \\ \mu_2 \\ \mu_3 \end{pmatrix} \text{, } \quad\Sigma = \begin{pmatrix} 1 &\rho &0 \\ \rho &1 &\rho \\ 0 &\rho &1 \end{pmatrix} \quad (\rho^2 < 1/2) \end{align}

What is the conditional distribution of $X_2$ given $X_1 = x_1$, $X_3 = x_3$? Under what condition does this distribution coincide with the marginal distribution of $X_2$?

I am sure that this is meant to be solved by the following theorem from the lecture:

If $x \sim \mathcal{N}(\mu,\Sigma)$ with $$ \mu = \begin{pmatrix} \mu_1 \\ \mu_2 \end{pmatrix}, \quad \Sigma = \begin{pmatrix} \Sigma_{11} &\Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \end{pmatrix}$$ then the conditional distribution of any subset vector $x_1$, given the complement vector $x_2$ is also a multivariate normal distribution given by $$x_1 \vert x_2 \sim \mathcal{N}(\mu_{1 \vert 2}, \Sigma_{1 \vert 2}),$$ where

$$\mu_{1 \vert 2} = \mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(x_2-\mu_2)$$ $$\Sigma_{1 \vert 2} = \Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$$

What I do not get here is how I should write down the $\mu$ and $\Sigma$ so that I can use the theorem from the lecture. Could you please give me a hint?

Answer 1

The key is "subset vectors". Here the subset vectors are $\begin{pmatrix} X_2\end{pmatrix}$ and its complement $\begin{pmatrix}X_1\\X_3\end{pmatrix}$ .

To better visualise the corresponding $\mu$ and $\Sigma$ aspects of these, consider this reordering: $$\mu'=\begin{pmatrix}\mu_2\\\hdashline \mu_1\\ \mu_3\end{pmatrix}\qquad\Sigma' =\begin{pmatrix}\begin{array}{l:ll}\sigma_{2,2} & \sigma_{2,1} &\sigma_{2,3}\\\hdashline\sigma_{1,2} &\sigma_{1,1} &\sigma_{1,3}\\\sigma_{3,2}&\sigma_{3,1}&\sigma_{3,3}\end{array}\end{pmatrix}=\begin{pmatrix}\begin{array}{l:ll}1 & \rho &\rho\\\hdashline\rho &1 &0\\\rho&0&1\end{array}\end{pmatrix}$$

So $\mu'_1=(\mu_2)~, \mu'_2=\begin{pmatrix}\mu_1\\\mu_3\end{pmatrix}~,{\Sigma'_{1,1}}=(1) ~, {\Sigma'_{1,2}}=\begin{pmatrix}\rho&\rho\end{pmatrix}~,$ et cetera.

Put these into your equations:

$${μ'_{1|2}=μ'_1+{Σ'_{12}}{Σ'_{22}}^{-1}(x'_2−μ'_2)\\ {Σ'_{1|2}}={Σ'_{11}}−{Σ'_{12}}{Σ'_{22}}^{-1}{Σ'_{21}}}$$

---

You want $(X_2\mid X_1{=}x_1,X_3{=}x_3)\sim\mathcal N(\mu_{2\mid 1,3},\Sigma_{2\mid 1,3})$ where

$$\begin{align}\mu_{2\mid 1,3} &=(\mu_2)+\begin{pmatrix}\sigma_{2,1}&\sigma_{2,3}\end{pmatrix}\begin{pmatrix}\sigma_{1,1} &\sigma_{1,3}\\\sigma_{3,1} & \sigma_{3,3}\end{pmatrix}^{-1}\begin{pmatrix}x_1-\mu_1\\x_3-\mu_3\end{pmatrix}\\[4ex]\Sigma_{2\mid 1,3}&=(\sigma_{2,2})-\begin{pmatrix}\sigma_{2,1}&\sigma_{2,3}\end{pmatrix}\begin{pmatrix}\sigma_{1,1} &\sigma_{1,3}\\\sigma_{3,1} & \sigma_{3,3}\end{pmatrix}^{-1}\begin{pmatrix}\sigma_{1,2}\\\sigma_{3,2}\end{pmatrix}\end{align}$$