# Conditional Normal

$X,Y,Z$ are standard normal R.V.

What is the value of $\operatorname{E}[X|X+Y+Z=1]$ and $\operatorname{Var}(X|X+Y+Z=1)$?

I think the first one should be $1/3$ by symmetry but don't know how to tackle the second part.

Any thought on this? Thanks a lot!

-

I presume you're assuming $X$, $Y$, $Z$ independent. Note that $(X+Y+Z)/\sqrt{3}$, $(Y - Z)/\sqrt{2}$ and $(2X-Y-Z)/\sqrt{6}$ are uncorrelated (and therefore independent) standard normal random variables. Since $X = (X+Y+Z)/3 + (2X-Y-Z)/3$, $\text{Var}(X|X+Y+Z=1) = (1/9) \text{Var}(2X-Y-Z) = 2/3$.

-
How did you get the three uncorrelated standard normal R.V.:(X+Y+Z)/3‾‾√, (Y−Z)/2‾‾√ and (2X−Y−Z)/6‾‾√ ? –  BVFanZ Jan 3 '12 at 21:55
From the three orthonormal vectors $(1,1,1)/\sqrt{3}$, $(0,1,-1)/\sqrt{2}$, $(2,-1,-1)/\sqrt{6}$. –  Robert Israel Jan 3 '12 at 22:06
There was considerable room for choice in deciding which linear combinations of $X,Y,Z$ should be the second and third of the three independent normals above. That makes the choice somewhat arbitrary, so it should be possible to do without choosing at all. –  Michael Hardy Jan 3 '12 at 23:29
Actually there was no need to mention the $(Y-Z)/\sqrt{2}$. You just want to write $X = (X+Y+Z)/3 + (2X-Y-Z)/3$ and note that $X+Y+Z$ and $2X-Y-Z$ are independent... –  Robert Israel Jan 4 '12 at 2:52

\begin{align} 1 & = E(X+Y+Z\mid X+Y+Z=1) \\ \\ & = E(X\mid X+Y+Z=1) + E(Y\mid X+Y+Z=1) + E(Z\mid X+Y+Z=1). \end{align} If they're independent, you've got symmetry that justifies the conclusion that the three terms are equal. You can get by with much weaker hypotheses than independence if they justify symmetry.

With the variance I'll just assume independence and leave weaker hypotheses for another occasion. We have this bivariate normal distribution: $$\begin{bmatrix} X \\ X+Y+Z \end{bmatrix} \sim N\left( \begin{bmatrix} 0 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 & 1 \\ 1 & 3 \end{bmatrix} \right),$$ and hence $\operatorname{cor}(X,X+Y+Z)=1/\sqrt{3}$.

So how do we think about the bivariate normal distribution $$\begin{bmatrix} U \\ V \end{bmatrix} \sim N\left(\begin{bmatrix} 0 \\ 0 \end{bmatrix}, \begin{bmatrix} \sigma^2, & \rho\sigma\tau \\ \rho\sigma\tau, & \tau^2 \end{bmatrix} \right)$$ where $\sigma,\tau$ are standard deviations and $\rho$ is the correlation?

Conditional on $V$ being a certain number of standard deviations above its mean, $U$ is expected to be $\rho$ times that many standard deviations about its mean. Thus $$E(U\mid V) = \rho\sigma \frac V\tau = \frac{\rho\sigma\tau}{\tau^2} V.$$ By the law of total variance we have \begin{align} \sigma^2 = \operatorname{var}(U) & = E(\operatorname{var}(U \mid V)) + \operatorname{var}(E(U\mid V)) \tag{law of total variance} \\ \\ & = E(\operatorname{var}(U \mid V)) + \operatorname{var}\left( \frac{\rho\sigma\tau}{\tau^2} V \right) = E(\operatorname{var}(U \mid V)) + \rho^2\sigma^2. \end{align} If you accept "homscedasticity", i.e. the conditional variance of $U$ given $V$ does not depend on $V$, then you get $\operatorname{var}(U\mid V) = (1-\rho^2)\sigma^2$.

Applied to the particular distribution we're considering, this says $$\operatorname{var}(X\mid X+Y+Z) = \left(1 - \frac13\right)\cdot 1 = \frac23.$$

-
OK, I've done some editing, and I think I've made the answer neat and orderly. Maybe I'll add something about homoscedasticity later. –  Michael Hardy Jan 4 '12 at 0:40

If $A$ and $B$ are jointly normal random variables, then the conditional density of $A$ given $B = b$ is a normal density with mean and variance as follows: $$E[A\mid B = b] = \mu_A + \left.\left.\frac{\text{cov}(A,B)}{\sigma_B^2} \right(b - \mu_B\right), ~~ \text{var}(A\mid B = b) = \frac{\sigma_A^2\sigma_B^2 - (\text{cov}(A,B))^2}{\sigma_B^2}$$ where ${cov}(A,B)$ is the covariance of $A$ and $B$.

If $X$, $Y$, $Z$ are jointly normal, then $X$ and $X+Y+Z$ are also jointly normal with \begin{align*} E[X+Y+Z] &= \mu_X + \mu_Y + \mu_Z,\\ \text{var}(X+Y+Z) &= \sigma_X^2 + \sigma_Y^2+\sigma_Z^2 +2\text{cov}(X,Y)+2\text{cov}(X,Z)+2\text{cov}(Y,Z)\\ \text{cov}(X,X+Y+Z) &= \sigma_X^2 + \text{cov}(X,Y)+\text{cov}(X,Z) \end{align*} and plugging and chugging in the result of the previous paragraph gives the desired answers for $E[X\mid X+Y+Z=1]$ and $\text{var}(X\mid X+Y+Z=1)$. Of course, considerable simplifications occur in the special case when $X$, $Y$, and $Z$ are independent standard normal random variables. We have $E[X+Y+Z]=0$, $\text{var}(X+Y+Z)=3$, $\text{cov}(X,X+Y+Z)=1$, and so $$E[X\mid X+Y+Z=1] = \left.\left.\frac{1}{3}\right(1-0\right) = \frac{1}{3}, ~\text{var}(X\mid X+Y+Z=1) = \frac{1\times 3 - 1^2}{3} = \frac{2}{3}$$ exactly as found by others as well.

-