# Conditional expectation of a joint normal distribution

Let $X_1, X_2$ be jointly normal $N(\mu, \Sigma)$.

I know that in general, $\mathbb{E}[X_2|X_1]$ can be computed by integrating the conditional density, but in the case of jointly normal variables, it suffices to do a linear projection:

$\mathbb{E}[X_2 | \sigma(X_1)] = \mathbb{E}[X_2|\mathrm{span}(\mathbf{1}, X_1)] = \mu_2 + \frac{\mathrm{cov}(X_2, X_1)}{\mathrm{var}(X_1)} (X_1 - \mu_1)$

Is there a neat proof of this fact (one doesn't require doing any integrals)? Looking for references too.

I've found an answer that I'm happy with:

$$Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X$$ is jointly normal with $X$ and uncorrelated, hence independent.

Therefore

$$\mathbb{E}[Y|X] = \mathbb{E}[Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X + \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X| X] \\ = \mathbb{E}[Y - \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} X] +\frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)}X \\ = \mathbb{E} [Y] + \frac{\mathrm{cov}(X, Y)}{\mathrm{var}(X)} (X - \mathbb{E}[X])$$

• But why $Y-Xcov(X,Y)/var(X)$ is uncorrelated with X? Commented Apr 9, 2019 at 0:23
• @YibeiHe: you can use bilinearity of $\mathrm{cov}$ to show their covariance comes out to $0$ Commented Apr 10, 2019 at 13:11
• This is the solution when $(X,Y)$ have bivariate normal distribution. For the general multivariate normal case, where $X$ and $Y$ are jointly normal random vectors, the formula is analogous, with $\operatorname{Cov}(X,Y)/\operatorname{Var}(X)$ replaced by the product of the cross-covariance matrix of $X, Y$ and the inverse of the covariance matrix of $X$. For a derivation see math.stackexchange.com/a/2209484/215011 Commented Mar 10, 2021 at 22:51

Let $$(X_1, X_2) \sim MVN (\mu, \Sigma)$$, then recall that \begin{align} f_{X_2|X_1}(x_2) = \frac{f_{X_1, X_2}(x_1, x_2)}{f_{X_1}(x_1)} \, , \end{align} where $$f_{X_1,X_2}(x_1,x_2)=\\ \frac{1}{2\pi \sigma_1\sigma_2 \sqrt{1-\rho}}\exp\left( - \frac{1}{2(1-\rho^2)}\left(\frac{(x_1 - \mu_1)^2}{\sigma^2} + \frac{(x_2 - \mu_2)^2}{\sigma^2} - \frac{2\rho(x_1-\mu_1)(x_2 - \mu_2)}{\sigma_1\sigma_2} \right) \right),$$ and $$f_{X_1}(x) = \frac{1}{\sqrt{2\pi\sigma_1^2}}\exp\left( - \frac{(x_1-\mu_1)^2}{2\sigma_1^2} \right).$$

After some simple algebra and rearrengments you'll find that $$X_2|X_1 = x_1 \sim N\left( \mu_2 + \rho\frac{\sigma_2}{\sigma_1}(x_1 - \mu_1), (1- \rho^2)\sigma_2^2 \right).$$

• Interestingly this linear regression of $X_2$ and $X_1$. Commented Oct 25, 2019 at 16:24

Just as a complement to @user357269's answer, I will mainly focus on explaining why we have the coefficient $$\frac{cov(X_1,X_2)}{Var(X_1)}$$.

It starts by acknowledging the following result about Bivariate normal: $$(X_1,X_2)^T \sim N(\mu,\Sigma) \Rightarrow \exists$$ a lower triangular matrix $$A \in \mathbb{R}^{2 \times 2}$$, and $$Z_1, Z_2 \sim N(0,1) \; iid$$ such that $$(X_1,X_2)^T = A (Z_1,Z_2)^T + \mu$$.

Furthermore, we have $$AA^T = \Sigma =\pmatrix{Var(X_1) & cov(X_1,X_2)\\cov(X_1,X_2) & Var(X_2)}$$.

Using Cholesky decomposition to solve A, we get $$A=\pmatrix{\sqrt{Var(X_1)} & 0\\ \frac{cov(X_1,X_2)}{\sqrt{Var(X_1)}} & \sqrt{Var(X_2)-\frac{cov(X_1,X_2)^2}{Var(X_1)}}}$$.

So, we can express $$X_1 = \sqrt{Var(X_1)}Z_1 + \mu_1$$ and $$X_2=\frac{cov(X_1,X_2)}{\sqrt{Var(X_1)}} Z_1 + \sqrt{Var(X_2)-\frac{cov(X_1,X_2)^2}{Var(X_1)}} Z_2 + \mu_2$$.

Since, $$Z_1 \perp Z_2$$, calculating the conditional expectation becomes much easier: \begin{align*} &E[X_2|X_1]\\ &=E[\frac{cov(X_1,X_2)}{\sqrt{Var(X_1)}} Z_1 + \sqrt{Var(X_2)-\frac{cov(X_1,X_2)^2}{Var(X_1)}} Z_2 + \mu_2|\sqrt{Var(X_1)} Z_1+ \mu_1]\\ &=E[\frac{cov(X_1,X_2)}{\sqrt{Var(X_1)}} Z_1 + \sqrt{Var(X_2)-\frac{cov(X_1,X_2)^2}{Var(X_1)}} Z_2 + \mu_2|Z_1]\\ &=\frac{cov(X_1,X_2)}{\sqrt{Var(X_1)}} Z_1 + \mu_2 \\ &= \frac{cov(X_1,X_2)}{Var(X_1)}(X_1 - \mu_1) + \mu_2 \end{align*}