7
$\begingroup$

Wikipedia gives details on the conditional distribution of the multivariate normal:

If $\mu$ and $\Sigma$ are partitioned as follows

$\boldsymbol\mu = \begin{bmatrix} \boldsymbol\mu_1 \\ \boldsymbol\mu_2 \end{bmatrix}$

$\boldsymbol\Sigma = \begin{bmatrix} \boldsymbol\Sigma_{11} & \boldsymbol\Sigma_{12} \\ \boldsymbol\Sigma_{21} & \boldsymbol\Sigma_{22} \end{bmatrix} \quad$

then, the distribution of $x_1$ conditional on $x_2= a$ is multivariate normal $(x_1|x_2=a) \sim N(\bar{\mu}, \bar{\Sigma})$ where

$\bar{\boldsymbol\mu} = \boldsymbol\mu_1 + \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \left( \mathbf{a} - \boldsymbol\mu_2 \right) $

and covariance matrix

$\overline{\boldsymbol\Sigma} = \boldsymbol\Sigma_{11} - \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \boldsymbol\Sigma_{21}. $

How can I prove this result? Wikipedia cites Eaton, Morris L. (1983). Multivariate Statistics: a Vector Space Approach. John Wiley and Sons. pp. 116–117., but I don't have this book handy...

$\endgroup$
4
  • $\begingroup$ How come you did not even react to the answer below? $\endgroup$
    – Did
    Jul 14, 2014 at 18:25
  • $\begingroup$ @Did, I actually used it, forgot to accept though. Sorted now. $\endgroup$
    – Grzenio
    Jul 15, 2014 at 7:56
  • $\begingroup$ If someone needs only the derivation of mean and variance in an easy way, refer to this: stats.stackexchange.com/questions/30588/… $\endgroup$
    – Naetmul
    Jun 18, 2015 at 6:07
  • $\begingroup$ ..and this answer stats.stackexchange.com/a/270934/145128 derives the mean and variance and establishes that the conditional distribution is Gaussian. $\endgroup$
    – grand_chat
    Mar 31, 2020 at 23:59

1 Answer 1

10
$\begingroup$

Consider the following transformation $$\begin{align} Y_1 &=X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2 \\ Y_2 &=X_2\end{align}$$ Then $$\begin{pmatrix} Y_1 \\ Y_2 \end{pmatrix} \sim\mathcal{N_p}\left[\begin{pmatrix} \mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2 \\ \mu_2 \end{pmatrix} ,\begin{pmatrix} \bar \Sigma & 0 \\ 0 & \Sigma_{22} \end{pmatrix} \right] $$

Hence the PDF of $\begin{pmatrix} Y_1 \\ Y_2 \end{pmatrix} $ is $$\begin{align}n(y_1,y_2) &= n(y_1|\mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma)\cdot n(y_2| \mu_2,\Sigma_{22}) \end{align}$$ Since $Y_1 \sim \mathcal{N_{p1}}(\mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma)$ and $Y_2 \sim \mathcal{N_{p-p1}}(\mu_2,\Sigma_{22})$ independently, as $Cov(Y_1,Y_2)=0$.

The PDF of $\begin{pmatrix} X_1 \\ X_2 \end{pmatrix} $ will be obtained by replacing $y_1$ by $x_1-\Sigma_{12}\Sigma_{22}^{-1}x_2$ and $y_2$ by $x_2$. Note that here jacobian of the transformation is unity.

Hence the PDF of $\begin{pmatrix} X_1 \\ X_2 \end{pmatrix} $ is $$\begin{align}n(x_1,x_2) &= n((x_1-\Sigma_{12}\Sigma_{22}^{-1}x_2)|\mu_1-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma)\cdot n(x_2| \mu_2,\Sigma_{22}) \end{align}$$

Hence the conditional PDF of $X_1$ given $X_2=a$ is

$$\begin{align}f_{X_1|X_2=a}(x_1) &= \dfrac{n(x_1,a)}{n(a|\mu_2,\Sigma_{22})} \\ &=n(x_1-\Sigma_{12}\Sigma_{22}^{-1}a|\mu_2-\Sigma_{12}\Sigma_{22}^{-1}\mu_2, \bar \Sigma) \\ &= \dfrac{1}{(2 \pi)^{p/2}\sqrt{|\bar \Sigma|}}\exp\left( \frac{1}{2}\left[ (x_1-\mu_1- \Sigma_{12}\Sigma_{22}^{-1}(a-\mu_2))'\bar \Sigma^{-1}(x_1-\mu_1- \Sigma_{12}\Sigma_{22}^{-1}(a-\mu_2)) \right]\right) \end{align}$$

Hence $X_1|X_2=a \sim \mathcal{N_{p1}}(\bar \mu, \bar \Sigma)$.

$\endgroup$
1
  • $\begingroup$ why is Y1 and Y2 independent? $\endgroup$
    – smaillis
    Mar 6 at 23:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.