I came across the following expression:

$E[X \mid Y=y, Z_1, Z_2] = \frac{\sum_x x P(X=x, Y=y \mid Z_1,Z_2)} { P(Y=y \mid Z_1,Z_2)}$

But I have no idea how this simplification works because I am not sure how to deal with conditional expectation when its conditioned on several random variables like "$Y=y,Z_1,Z_2$" as above. Is it possible to just move the $Y=y$ over to the other side like that in a conditional expectation when you have several variables? Can someone please explain the inner workings?


What you have to recognize here is that both sides are random variables. They are functions. For example, the right hand side is a function, defined as : $E[X | Y = y,Z_1,Z_2](z_1,z_2) = E[X | Y = y,Z_1 = z_1, Z_2 = z_2]$.

So how do you check if two random variables are equal? Of course, by checking if they are equal at each point. So fix a point $(z_1,z_2)$ at which both sides are being evaluated.

The left hand side is $E[X | Y = y, Z_1 = z_1 , Z_2 = z_2]$, which can be written as : $$ \sum_{x} xP(X = x | Y = y , Z_1 = z_1, Z_2 = z_2) $$

This expands to, by the usual rule : $$ \sum_{x} \frac{xP(x=X,Y=y,Z_1 = z_1, Z_2 = z_2)}{P(Y = y , Z_1 = z_1,Z_2 = z_2)} $$

Multiply and divide by $P(Z_1 = z_1,Z_2 = z_2)$ : $$ \sum_{x} x\frac{P(X=x,Y=y,Z_1 = z_1,Z_2 =z_2) / P(Z_1 = z_1, Z_2 = z_2)}{P(Y= y, Z_1 = z_1,Z_2 = z_2)/P(Z_1 = z_1,Z_2 = z_2)} $$

which simplifies via conditional expectation to : $$ \sum_{x} x \frac{P(X= x, Y = y|Z_1 = z_1, Z_2 = z_2)}{P(Y = y | Z_1 = z_1, Z_2 = z_2)} $$

which is nothing but by definition : $$ \left[\sum_{x} x\frac{P(X=x,Y=y|Z_1,Z_2)}{P(Y=y,Z_1,Z_2)}\right] (z_1,z_2) $$

because the application of the function is the same as what happens when $Z_1 = z_1$ and $Z_2 = z_2$.

Since these functions are equal at all points, they are equal,hence the result follows.

To deal then, with multiple variables, you need to recognize whether you are dealing with an equality of numbers or of functions, like here. Imagine you are setting values for all the conditioned variables, and then you can push those in the right of the $|$ to the left of the $|$ like how I did with the $Y$ variable : by dividing and multiplying by an appropriate term. Finally, whatever is left reflects a functional equality, as has happened here.

This trick will of course have a different version once you deal with conditioning over sigma-algebras, like in more rigorous probability.

| cite | improve this answer | |
  • $\begingroup$ Thanks. A small typo in the second equation, where it should read "P(X = x, ...". Do you recommend any notes/book that goes a bit deeper into such complicated conditional expectation expressions? Or does it always get worked out from first principles? $\endgroup$ – cbro Feb 1 at 9:19
  • $\begingroup$ Nah, pretty much everywhere it gets worked out like this. I struggled and got used to it only because I work in probability so have to see this kind of thing often. But if you post your query onto MSE, then you will always get the details, or at least a more specific overview than the book , maybe. $\endgroup$ – Teresa Lisbon Feb 1 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.