Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Wikipedia gives details on the conditional distribution of the multivariate normal:

If $\mu$ and $\Sigma$ are partitioned as follows

$\boldsymbol\mu = \begin{bmatrix} \boldsymbol\mu_1 \\ \boldsymbol\mu_2 \end{bmatrix}$

$\boldsymbol\Sigma = \begin{bmatrix} \boldsymbol\Sigma_{11} & \boldsymbol\Sigma_{12} \\ \boldsymbol\Sigma_{21} & \boldsymbol\Sigma_{22} \end{bmatrix} \quad$

then, the distribution of $x_1$ conditional on $x_2= a$ is multivariate normal $(x_1|x_2=a) \sim N(\bar{\mu}, \bar{\Sigma})$ where

$\bar{\boldsymbol\mu} = \boldsymbol\mu_1 + \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \left( \mathbf{a} - \boldsymbol\mu_2 \right) $

and covariance matrix

$\overline{\boldsymbol\Sigma} = \boldsymbol\Sigma_{11} - \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \boldsymbol\Sigma_{21}. $

How can I prove this result? Wikipedia cites Eaton, Morris L. (1983). Multivariate Statistics: a Vector Space Approach. John Wiley and Sons. pp. 116–117., but I don't have this book handy...

share|improve this question
    
How come you did not even react to the answer below? –  Did Jul 14 at 18:25
    
@Did, I actually used it, forgot to accept though. Sorted now. –  Grzenio Jul 15 at 7:56

1 Answer 1

up vote 3 down vote accepted

Consider the following transformation $$\begin{align} Y_1 &=X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2 \\ Y_2 &=X_2\end{align}$$ Then $$\begin{pmatrix} Y_1 \\ Y_2 \end{pmatrix} \sim\mathcal{N_p}\left[\begin{pmatrix} \mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2 \\ \mu_2 \end{pmatrix} ,\begin{pmatrix} \bar \Sigma & 0 \\ 0 & \Sigma_{22} \end{pmatrix} \right] $$

Hence the PDF of $\begin{pmatrix} Y_1 \\ Y_2 \end{pmatrix} $ is $$\begin{align}n(y_1,y_2) &= n(y_1|\mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma)\cdot n(y_2| \mu_2,\Sigma_{22}) \end{align}$$ Since $Y_1 \sim \mathcal{N_{p1}}(\mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma)$ and $Y_2 \sim \mathcal{N_{p-p1}}(\mu_2,\Sigma_{22})$ independently, as $Cov(Y_1,Y_2)=0$.

The PDF of $\begin{pmatrix} X_1 \\ X_2 \end{pmatrix} $ will be obtained by replacing $y_1$ by $x_1-\Sigma_{12}\Sigma_{22}^{-1}x_2$ and $y_2$ by $x_2$. Note that here jacobian of the transformation is unity.

Hence the PDF of $\begin{pmatrix} X_1 \\ X_2 \end{pmatrix} $ is $$\begin{align}n(x_1,x_2) &= n((x_1-\Sigma_{12}\Sigma_{22}^{-1}x_2)|\mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma)\cdot n(x_2| \mu_2,\Sigma_{22}) \end{align}$$

Hence the conditional PDF of $X_1$ given $X_2=a$ is

$$\begin{align}f_{X_1|X_2=a}(x_1) &= \dfrac{n(x_1,a)}{n(a|\mu_2,\Sigma_{22})} \\ &=n(x_1-\Sigma_{12}\Sigma_{22}^{-1}a|\mu_2-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma) \\ &= \dfrac{1}{(2 \pi)^{p/2}\sqrt{|\bar \Sigma|}}\exp\left( \frac{1}{2}\left[ (x_1-\mu_1- \Sigma_{12}\Sigma_{22}^{-1}(a-\mu_2))'\bar \Sigma^{-1}(x_1-\mu_1- \Sigma_{12}\Sigma_{22}^{-1}(a-\mu_2)) \right]\right) \end{align}$$

Hence $X_1|X_2=a \sim \mathcal{N_{p1}}(\bar \mu, \bar \Sigma)$.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.