I am studying conditional expectations and am stuck with this issue. Consider an expectation conditional on a fixed sum of random variables, e.g. $E[\; X \; |\; X+Y = \alpha]$. Then we can retrieve it from joint density $f(x,y)$ as in here.

What if our conditional part is more complicated such as a linear combination of random variables? What can we say about

$$E[X_i \mid a_1 X_1 + a_2 X_2 + \dots + a_n X_n = \beta] ?$$

Can we get a nice formula as in the case of two random variables? It seems that the joint density won't be enough, what else is needed? I'm hesitating here since we cannot rewrite $X_j$ as a function of $X_i$-only anymore as in the previous case.

If possible, I'd like it to be quite general but you may simplify all this with additional assumptions if it's easier to write it out. Are there any references where I could read more about this particular problem?

Thank you.


Of course the joint density is enough (the joint density is fully informative). In general,

$$E[X_1|g(X_1,X_2 \cdots)=c] = \frac{\int_{{\bf x} \in A} X_1 f({\bf x})d{\bf x}}{\int_{{\bf x} \in A} f({\bf x}) d{\bf x}}$$

where $A$ is the region where $g(X_1,X_2 \cdots)=c$. (In you particular case, this region is a hyperplane)

Added: In the particular case in which $X_1,X_2 \cdots$ are jointly gaussian, letting $Z=a_1 X_1 + a_2 X_2 + \dots + a_n X_n = {\bf a^T X}$, then $X,Z$ are also jointly gaussian and

$$E(X_1|Z=z)=E[X_1]- \frac{Cov(X_1,Z)}{Var(Z)}(z-E[Z])$$

with $E[Z] = {\bf a^T}E[{\bf X}]$, $Var(Z)={\bf a^T}{\bf \Sigma_X}{\bf a}$ and $Cov(X_1,Z)=[1,0,0\cdots,0]\; {\bf \Sigma_X} \; {\bf a}=a_1 Var(X_1)+a_2 Cov(X_1 X_2) + \cdots$

This expression can also be regarded (when the variables are not guaranteed to be jointly gaussian) as a linear approximation of the true expectation - which, in general would be non linear.

  • $\begingroup$ Thank you. This generalized notation makes things actually much clearer. $\endgroup$ – johnny Mar 21 '12 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.