Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Does anyone know how to start this question?

Let random vectors $x,u,v$ have joint Gaussian distribution, and $u,v$ be independent. Show that $E(x|u,v)=E(x|u)+E(x|v)-E(x)$.

share|cite|improve this question
Write $x=au+bv+w$ for some numbers $a$ and $b$, with $(u,v,w)$ gaussian and independent. – Did Nov 3 '11 at 11:38
Are random vectors the same as random variables? or do $x,u,v$ all have multivariate jointly Gaussian distributions (that is, the components of each are not necessarily independent Gaussian random variables) but the joint distribution of $u$ and $v$ factors into the product of the jointly Gaussian distributions of vectors $u$ and of $v$? – Dilip Sarwate Nov 3 '11 at 16:35
"Are random vectors the same as random variables?" I really want to assume they are the same here. – kkk Nov 4 '11 at 7:12
A random vector is simply a vector-valued random variable. – Michael Hardy Nov 4 '11 at 22:01

Write $x=au+bv+w$ for some numbers $a$ and $b$, with $(u,v,w)$ gaussian and independent.

Thus $y=E(x|u,v)$ is $y=au+bv+E(w)$. Since $(u,v)$ is independent, \begin{align} & E(x|u) = E(y|u) = au+bE(v)+E(w), \\ & E(x|v) = E(y|v) = aE(u)+bv+E(w) \\ & E(x) = E(y) = aE(u)+bE(v)+E(w). \end{align} This yields \begin{align} E(x|u)+E(x|v) & = au+aE(u)+bv+bE(v)+2E(w) \\ & = y+aE(u)+bE(v)+E(w) \end{align} hence $$ E(x|u)+E(x|v)=E(x|u,v)+E(x). $$

share|cite|improve this answer
Who says you can write $x = au + bv + w$ for some $a$ and $b$? – TMM Jan 13 '13 at 19:47
@TMM: Who says...? Well, the definition of being multivariate gaussian... – cardinal Jan 13 '13 at 23:22
@cardinal It's probably me, but I still don't see it. If $X,Y$ are random variables and $X$ depends on $Y$, you cannot say $X = aY + Z$ either (with $Z$ independent from $Y$). – TMM Jan 14 '13 at 0:00
@TMM: Use the linearity property in the definition and note that if $U$ and $V$ are independent then they are generated from orthogonal vectors corresponding to a subspace of dimension two. Consider an orthonormal basis extension to the full space in this coordinate system. Everything should then be clear. – cardinal Jan 14 '13 at 0:29
@TMM If $(x,y)$ is gaussian, then there exists some (explicit) coefficients $(a,b)$ such that $E(x\mid y)=ay+b$. You are right that this is a specific property of gaussian families, which does not hold in general. – Did Jan 14 '13 at 9:36

How far $u$ is above the expected value $\mu_u$ of $u$ is $\dfrac{x-\mu_u}{\sigma_u}$ where $\sigma_u$ is the standard deviation of $u$.

How many $x$-standard deviations one should "expect" $x$ to be above the expected value of $x$, given the observed value of $u$, is that expression above multiplied by the correlation between $x$ and $u$; thus it is $\rho_{x,u}\left(\dfrac{x-\mu_u}{\sigma_u}\right)$. So $$ E(x\mid u) = \mu_x + \sigma_x \rho_{x,u}\left(\dfrac{x-\mu_u}{\sigma_u}\right) $$ where $\mu_x$ and $\sigma_x$ are the expected value and standard deviation of $x$ respectively. And of course a similar result applies to $v$.

So $$ \begin{align} & {} \quad E(X\mid u) + E(x\mid c) - E(x) \\ \\ & = \mu_x + \sigma_x \rho_{x,u}\left(\frac{x-\mu_u}{\sigma_u}\right) + \mu_x + \sigma_x \rho_{x,v}\left(\frac{x-\mu_v}{\sigma_v}\right) - \mu_x \\ \\ & = \mu_x + \sigma_x \rho_{x,u}\left(\frac{x-\mu_u}{\sigma_u}\right) + \sigma_x \rho_{x,v}\left(\frac{x-\mu_v}{\sigma_v}\right) \\ \\ \\ & = \mu_x + \frac{\rho_{x,u}\sigma_x\sigma_u}{\sigma_u^2}\left(x-\mu_u\right) + \frac{\rho_{x,v}\sigma_x\sigma_v}{\sigma_u^2}\left(x-\mu_v\right) \\ \\ \\ & = \mu_x + \frac{\operatorname{cov}(x,u)}{\sigma_u^2}\left(x-\mu_u\right) + \frac{\operatorname{cov}(x,v)}{\sigma_u^2}\left(x-\mu_v\right) \end{align} $$

The whole matrix of covarinaces, which is the variance of the random vector $(x,u,v)^\top$ is $$ \begin{bmatrix} \sigma_x^2, & \rho_{x,u}\sigma_x\sigma_u, & \rho_{x,v}\sigma_x\sigma_v \\ \rho_{x,u}\sigma_x\sigma_u, & \sigma_u^2, & 0 \\ \rho_{x,v}\sigma_x\sigma_v, & 0,& \sigma_v^2 \end{bmatrix}. $$ Via the usual formulas for conditional expectations in a multivariate normal distribution, we have $$ E\left(x \mid \begin{bmatrix} u \\ v \end{bmatrix}\right) = \mu_x + \begin{bmatrix} \operatorname{cov}(x,u), & \operatorname{cov}(x,v) \end{bmatrix} \begin{bmatrix} \sigma_u^2 & 0 \\ 0 & \sigma_v^2 \end{bmatrix}^{-1} \begin{bmatrix} u-\mu_u \\ v-\mu_v \end{bmatrix}. $$

Since the matrix is diagonal, the inversion is simple: just invert the diagonal elements. Multiply the matrices, and you have the desired result.

If $u$ and $v$ had not been independent, this would have been more complicated.

Later edit: I see I treated these three things as scalar-valued. If they're vectors, and not necessarily all of the same number of scalar components, then where I wrote $\dfrac{\operatorname{cov}(x,u)}{\sigma_u^2}$, we would need $\operatorname{cov}(x,u)(\sigma_u^2)^{-1}$ where the two things being multiplied are matrices, in that order. Otherwise it's the same.

share|cite|improve this answer
Even if $x$,$u$,$,v$ are vectors as indicated in the questions, we could still write $E(x)$ as a scalar by thinking $x$ is some random variables taking different values? – kkk Nov 4 '11 at 7:12
@kkk I'm not sure I understand your last question. Certainly every random vector has components that are scalar-valued random variables. And their joint distribution amounts to the distribution of the random vector. – Michael Hardy Nov 4 '11 at 22:00

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.