Let $X_1, X_2$ be jointly normal $N(\mu, \Sigma)$.

I know that in general, $\mathbb{E}[X_2|X_1]$ can be computed by integrating the conditional density, but in the case of jointly normal variables, it suffices to do a linear projection:

$\mathbb{E}[X_2 | \sigma(X_1)] = \mathbb{E}[X_2|\mathrm{span}(\mathbf{1}, X_1)] = \mu_2 + \frac{\mathrm{cov}(X_2, X_1)}{\mathrm{var}(X_1)} (X_1 - \mu_1) $

Is there a neat proof of this fact (one doesn't require doing any integrals)? Looking for references too.


I've found an answer that I'm happy with:

$$ Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X $$ is jointly normal with $X$ and uncorrelated, hence independent.


$$\mathbb{E}[Y|X] = \mathbb{E}[Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X + \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X| X] \\ = \mathbb{E}[Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X] +\frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)}X \\ = \mathbb{E} [Y] + \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} (X - \mathbb{E}[X])$$

  • $\begingroup$ Definitely the bast approach, +1. $\endgroup$ – Did Mar 3 '17 at 6:29
  • $\begingroup$ But why $Y-Xcov(X,Y)/var(X)$ is uncorrelated with X? $\endgroup$ – Yibei He Apr 9 at 0:23
  • $\begingroup$ @YibeiHe: you can use bilinearity of $\mathrm{cov}$ to show their covariance comes out to $0$ $\endgroup$ – user357269 Apr 10 at 13:11

Let $(X_1, X_2) \sim MVN (\mu, \Sigma)$, then recall that \begin{align} f_{X_2|X_1}(x_2) = \frac{f_{X_1, X_2}(x_1, x_2)}{f_{X_1}(x_1)} \, , \end{align} where $$ f_{X_1,X_2}(x_1,x_2)=\\ \frac{1}{2\pi \sigma_1\sigma_2 \sqrt{1-\rho}}\exp\left( - \frac{1}{2(1-\rho^2)}\left(\frac{(x_1 - \mu_1)^2}{\sigma^2} + \frac{(x_2 - \mu_2)^2}{\sigma^2} - \frac{2\rho(x_1-\mu_1)(x_2 - \mu_2)}{\sigma_1\sigma_2} \right) \right), $$ and $$ f_{X_1}(x) = \frac{1}{\sqrt{2\pi\sigma_1^2}}\exp\left( - \frac{(x_1-\mu_1)^2}{2\sigma_1^2} \right). $$

After some simple algebra and rearrengments you'll find that $$ X_2|X_1 = x_1 \sim N\left( \mu_2 + \rho\frac{\sigma_2}{\sigma_1}(x_1 - \mu_1), (1- \rho^2)\sigma_2^2 \right). $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.