11
$\begingroup$

Let $X_1, X_2$ be jointly normal $N(\mu, \Sigma)$.

I know that in general, $\mathbb{E}[X_2|X_1]$ can be computed by integrating the conditional density, but in the case of jointly normal variables, it suffices to do a linear projection:

$\mathbb{E}[X_2 | \sigma(X_1)] = \mathbb{E}[X_2|\mathrm{span}(\mathbf{1}, X_1)] = \mu_2 + \frac{\mathrm{cov}(X_2, X_1)}{\mathrm{var}(X_1)} (X_1 - \mu_1) $

Is there a neat proof of this fact (one doesn't require doing any integrals)? Looking for references too.

$\endgroup$

2 Answers 2

Reset to default
16
$\begingroup$

I've found an answer that I'm happy with:

$$ Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X $$ is jointly normal with $X$ and uncorrelated, hence independent.

Therefore

$$\mathbb{E}[Y|X] = \mathbb{E}[Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X + \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X| X] \\ = \mathbb{E}[Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X] +\frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)}X \\ = \mathbb{E} [Y] + \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} (X - \mathbb{E}[X])$$

$\endgroup$
4
  • 1
    $\begingroup$ Definitely the bast approach, +1. $\endgroup$
    – Did
    Mar 3, 2017 at 6:29
  • $\begingroup$ But why $Y-Xcov(X,Y)/var(X)$ is uncorrelated with X? $\endgroup$
    – clement
    Apr 9, 2019 at 0:23
  • $\begingroup$ @YibeiHe: you can use bilinearity of $\mathrm{cov}$ to show their covariance comes out to $0$ $\endgroup$
    – user357269
    Apr 10, 2019 at 13:11
  • 1
    $\begingroup$ This is the solution when $(X,Y)$ have bivariate normal distribution. For the general multivariate normal case, where $X$ and $Y$ are jointly normal random vectors, the formula is analogous, with $\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$ replaced by the product of the cross-covariance matrix of $X, Y$ and the inverse of the covariance matrix of $X$. For a derivation see math.stackexchange.com/a/2209484/215011 $\endgroup$
    – grand_chat
    Mar 10, 2021 at 22:51
4
$\begingroup$

Let $(X_1, X_2) \sim MVN (\mu, \Sigma)$, then recall that \begin{align} f_{X_2|X_1}(x_2) = \frac{f_{X_1, X_2}(x_1, x_2)}{f_{X_1}(x_1)} \, , \end{align} where $$ f_{X_1,X_2}(x_1,x_2)=\\ \frac{1}{2\pi \sigma_1\sigma_2 \sqrt{1-\rho}}\exp\left( - \frac{1}{2(1-\rho^2)}\left(\frac{(x_1 - \mu_1)^2}{\sigma^2} + \frac{(x_2 - \mu_2)^2}{\sigma^2} - \frac{2\rho(x_1-\mu_1)(x_2 - \mu_2)}{\sigma_1\sigma_2} \right) \right), $$ and $$ f_{X_1}(x) = \frac{1}{\sqrt{2\pi\sigma_1^2}}\exp\left( - \frac{(x_1-\mu_1)^2}{2\sigma_1^2} \right). $$

After some simple algebra and rearrengments you'll find that $$ X_2|X_1 = x_1 \sim N\left( \mu_2 + \rho\frac{\sigma_2}{\sigma_1}(x_1 - \mu_1), (1- \rho^2)\sigma_2^2 \right). $$

$\endgroup$
1
  • $\begingroup$ Interestingly this linear regression of $X_2$ and $X_1$. $\endgroup$
    – Ramesh Kadambi
    Oct 25, 2019 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.