# Conditional expectation of random variable parameterized by random variable

Suppose $$M(\lambda)$$ is a distribution that satisfies for all $$Z\sim M(\lambda)$$ that $$E[Z]=\lambda$$. Does it follow that if $$X \sim M(Y)$$ then $$E[X | Y] = Y$$ a.s.? Intuitively this seems to be the case, but how does one prove it, if it is true?

Since this has been cause of a lot of confusion; some clarification:

Fix a probability space $$(\Omega,\mathcal F, P)$$ so that it can accommodate the following random variables. There is only this probability space involved.

$$Y$$ is a random variable with expectation that takes values in $$\mathbb R$$ and for every $$\lambda\in\mathbb R$$, $$M(\lambda)$$ is a distribution on $$\mathbb R$$.

Suppose $$Q_\lambda$$ is the measure with distribution $$M(\lambda)$$. I am assuming $$Q_\lambda$$ to be defined on the Borel sets of $$\mathbb R$$. I am defining $$X\sim M(Y)$$ to mean that $$E[1_{\{X\in A\}}|Y] = P(X\in A | Y) = Q_Y(A)$$ for all measurable $$A\subseteq\mathbb R$$ $$P$$-a.s.

The conditional expectation $$E[X | Y]$$ is then defined as usual, as the a.s. uniquely determined random variable $$H$$ satisfying $$E[ZH] = E[ZX]$$ for all $$\sigma(Y)$$-measurable bounded $$Z$$.

• I don't understand the downvotes. Can someone explain? Commented Nov 11, 2022 at 18:06
• It would make sense to talk about conditional distributions: $X|Y=y\sim M(y)$ for a distribution $M$ with parameter $y$ equal to its mean. Is this the context you are looking for? In any case, add context and attempts to your question. Commented Nov 11, 2022 at 18:21
• Yes, this is the context I am looking for. I haven't included attempts because I haven't had any mentionworthy progress. I tried the standard things like plugging it into the definition of conditional expectation or considering the difference. I don't think that adds much to my question. I was thinking that this should be obvious and not have a very lengthy or hard proof, which is why I asked here. Commented Nov 11, 2022 at 18:33
• You should probably use the notation $X\mid Y\sim M(Y)$. The distribution of $X$ is the measure $\mu$ where $\mu(A) = E[M(Y,A)]$. (Also, you can drop the $Q_\lambda$ notation. A distribution is a measure, so $Q_\lambda=M(\lambda)$. Just use $M(\lambda,A)$ as shorthand for $M(\lambda)(A)$.) Commented Nov 13, 2022 at 14:15

You say that for each $$\lambda\in\mathbb{R}$$, $$M(\lambda)$$ is a distribution on $$\mathbb{R}$$, by which I assume you mean a Borel probability measure. Let $$\mathcal{R}$$ denote the Borel $$\sigma$$-algebra on $$\mathbb{R}$$. Let us further suppose that for every $$A\in\mathcal{R}$$, the function $$M(\cdot,A)$$ is Borel measurable. Then $$M$$ is what's called a probability kernel from $$\mathbb{R}$$ to $$\mathbb{R}$$. It follows that $$M(Y)$$ is a random measure, meaning $$M(Y(\omega))$$ is a measure for each $$\omega$$ and $$M(Y(\cdot), A)$$ is a random variable for each $$A$$. It can be show that this implies $$M(Y)$$ is a measurable function from $$\Omega$$ to $$M_1(\mathbb{R})$$, the space of Borel probability measures on $$\mathbb{R}$$, where we equip $$M_1$$ with the $$\sigma$$-algebra generated by the projection maps, $$\sigma(\{\pi_A: A \in \mathcal{R}\})$$. (Here, $$\pi_A$$ is the map $$\nu\mapsto \nu(A)$$.)

A regular conditional distribution for $$X$$ given $$\mathcal{G}$$ is a random measure $$\mu$$ such that $$P(X\in A\mid \mathcal{G})=\mu(\cdot,A)$$ a.s. for each $$A$$. Your hypothesis assert that $$M(Y)$$ is a regular conditional distribution for $$X$$ given $$Y$$ (strictly speaking, given $$\sigma(Y)$$).

A special case of Theorem 5.4 in Foundations of Modern Probability by Olav Kallenberg states that if $$\mu$$ is a regular conditional distribution for $$X$$ given $$Y$$ and $$f:\mathbb{R}^2\to\mathbb{R}$$ is measurable with $$E|f(X,Y)|<\infty$$, then $$E[f(X,Y) \mid Y](\omega) = \int_\mathbb{R} f(x,Y)\,\mu(\omega,dx)$$ for $$P$$-almost every $$\omega\in\Omega$$. This can be proven as follows. If $$f=1_{A\times B}$$, it follows easily from the hypotheses. Use the $$\pi$$-$$\lambda$$ theorem to prove it for all indicators $$f=1_C$$. Use linearity of conditional expectation to prove it for all simple $$f$$. Use the monotone convergence theorem for conditional expectations to prove it for all nonnegative $$f$$. Then apply this to the positive and negative parts of $$f$$.

In your case, since $$\int x\,M(y,dx)=y$$ for all $$y$$, the above yields $$E[X \mid Y] = \int_\mathbb{R} x \, M(Y,dx) = Y \text{ a.s.,}$$ which is what you were hoping to conclude.

EDIT: See below for a response to your clarification.

Also, the notation in my original answer doesn't match the notation of your clarification, but if I try to reconcile them I'll probably introduce a typo. So I'm leaving the original answer as is.

SECOND EDIT: We've been talking past each other this whole time! I think I've figured it out, though.

I think there is some confusion here, so first I'm going to restate your question.

Let $$(\Omega, \mathcal{F}, \mu)$$ be a probability space. Suppose that $$M(\lambda)$$ is a family of distributions on $$\Omega$$ parametrized by $$\lambda \in \Lambda$$. For example, perhaps $$\Omega = \mathbf{R}$$, $$\lambda \in \mathbf{R}$$, and $$M(\lambda)$$ is $$N(\lambda, 1)$$, the unit variance normal distribution centered on $$\lambda$$; or perhaps $$\Omega = \mathbf{R}_+$$, $$\lambda \in \mathbf{R}_+$$, and $$M(\lambda)$$ is an exponential distribution with scale $$\lambda$$; or perhaps $$\Omega = \mathbf{N}$$, $$\lambda \in \mathbf{N}$$, and $$M(\lambda)$$ is Poisson with rate $$\lambda$$; and so on. The hypothesis made in the question is that the mean of an $$M(\lambda)$$-distributed random variable is $$\lambda$$. That is, $$\Lambda \subseteq \Omega$$, and if $$Z \sim M(\lambda)$$, then $$\mathbf{E}[Z] = \lambda$$ (briefly, the family is indexed by the distribution's mean).

We will need $$\Lambda$$ to be a probability space. For this, we will assume $$\Lambda = \Omega$$ and that $$\Lambda$$ is endowed with same $$\sigma$$-algebra $$\mathcal{F}$$ as $$\Omega$$ and same measure $$\mu$$.

Suppose that $$(\Theta, \mathcal{G}, \nu)$$ is a probability space and that $$Y$$ is a random variable on $$\Theta$$ with values in $$\Lambda = \Omega$$. (We need the $$\sigma$$-algebra on $$\Lambda$$ to make sense of this.) The value of $$Y$$ determines a distribution $$M(Y)$$. This is close to saying that $$M(Y)$$ is a random distribution. However, a technical point intervenes here: For it to be a a random distribution, $$M$$ needs to be measurable. By endowing $$\mathfrak{M}_1(\Omega)$$, the usual space of probability measures on $$\Omega$$, with a $$\sigma$$-algebra, we can make sense of this assumption. (This is necessary to forbid pathological situations. If $$\Omega = \mathbf{R}$$, $$S \subseteq \mathbf{R}$$ is a non-measurable set, $$M(\lambda)$$ is $$N(\lambda, 1)$$ if $$\lambda \in S$$, and $$M(\lambda)$$ is $$N(\lambda, 2)$$ if $$\lambda \not\in S$$, then $$X$$ isn't measurable.) Then $$M(Y)$$, which means $$M \circ Y$$, is a random probability measure. The event space of $$M(Y)$$ is $$\Theta$$ and the values are in $$\mathfrak{M}_1(\Omega)$$.

The question is: Suppose that $$X = M(Y)$$. Is $$\mathbf{E}[X|Y] = Y$$ almost surely? Intuitively, the mean of $$X$$ is the value of $$Y$$, so this perhaps a reasonable-looking formula.

The answer is no, because if you look at the formula a little closer it turns out to be entirely unreasonable. We have random variables $$Y \colon \Theta \to \Omega$$ and $$M \colon \Omega \to \mathfrak{M}_1(\Omega)$$. By definition, $$X = M \circ Y \colon \Theta \to \mathfrak{M}_1(\Omega)$$. Since $$X$$ takes values in $$\mathfrak{M}_1(\Omega)$$, $$\mathbf{E}[X|Y]$$ must be a random probability measure on $$\Omega$$. It can't equal $$Y$$ because $$Y$$ is a random element of $$\Omega$$. In fact, because $$X = M \circ Y$$ is $$\sigma(Y)$$-measurable we have $$\mathbf{E}[X|Y] = \mathbf{E}[X|\sigma(Y)] = X$$.

(If you want to be careful about this, then you need to define $$\mathbf{E}[X|Y]$$; for this, recall that $$\mathfrak{M}_1(\Omega)$$ is contained in the topological vector space of signed measures on $$\Omega$$, and a topological vector space is a reasonable place in which to take expectations.)

So as stated, the question is asking about the equality of two objects of different types and is categorically false. However, there is a way of repairing this: We simply take a different expectation. For clarity we write expectation on $$\Omega$$ as $$\mathbf{E}_\Omega$$ and expectation on $$\Theta$$ as $$\mathbf{E}_\Theta$$. Remember we assumed that if $$Z \sim M(\lambda)$$, then $$\mathbf{E}_\Omega[Z] = \lambda$$. We may rewrite this as $$\mathbf{E}_\Omega \circ M = 1_\Omega$$, where $$1_\Omega$$ is the identity function on $$\Omega$$. It is then trivially true that $$\mathbf{E}_\Omega \circ M \circ Y = Y$$. Consequently, $$\mathbf{E}_\Omega \circ X = Y$$, or in more conventional notation, $$\mathbf{E}_\Omega[X] = Y.$$ It's very tempting to write this as $$\mathbf{E}[X] = Y$$, but $$\mathbf{E}$$ runs the risk of being interpreted as $$\mathbf{E}_\Theta$$, and as discussed above, that's a type error. To recover a result a little more like the one you're interested in, you can apply $$\mathbf{E}_\Theta[\cdot|Y]$$ to both sides to get $$\mathbf{E}_\Theta[\mathbf{E}_\Omega[X]|Y] = Y.$$

I believe my original answer is correct. Here is what I make of your clarification.

You have, in the notation of my original answer, $$\Lambda = \Omega = \mathbf{R}$$; your $$\Omega$$ was my $$\Theta$$. I'll use your notation from here on, though. For each $$\lambda \in \mathbf{R}$$, there is a measure $$Q_\lambda$$, and $$M(\lambda) = Q_\lambda$$.

You define $$X \sim M(Y)$$ to mean that $$X$$ is a random variable such that $$\mathbf{P}[X \in A|Y]$$. Fix $$\lambda \in \mathbf{R}$$. For the sake of argument, suppose it's valid to condition on the event $$Y = \lambda$$ and that probability zero events don't happen. In that case, the only thing $$\mathbf{P}[X \in A|Y = \lambda]$$ could be is $$Q_\lambda(A)$$, and that means $$\mathbf{E}[X|Y = \lambda]$$ is a random variable with distribution $$Q_\lambda$$. In particular, it sounds like the event space of this random variable is $$\mathbf{R}$$. It follows that the event space of $$X$$ must be $$\Omega \times \mathbf{R}$$: In order to actually sample from $$X$$, first you sample an $$\omega \in \Omega$$, and second you use $$Q_{Y(\omega)}$$ to sample from $$\mathbf{R}$$.

Sampling $$\omega \in \Omega$$ determines $$Q_{Y(\omega)} \in \mathfrak{M}_1(\mathbf{R})$$. It follows that $$Q_Y$$ is a random measure on $$\mathbf{R}$$. It's simply the composite $$\omega \mapsto Y(\omega) \mapsto Q_{Y(\omega)}$$. The measure $$Q_{Y(\omega)}$$ is the law of a random variable, and in my original answer I conflated the random variable with its law. It's this identification that allows me to write $$X = M \circ Y$$, as follows.

It's true that you wrote $$X \sim M(Y)$$, not $$X = M(Y)$$. But $$X \sim M(Y)$$ means, "$$X$$ is a random variable with distribution $$M(Y)$$." Since $$X$$ and $$M(Y)$$ have the same event space, as functions from the event space to the codomain, $$X$$ is a.s. equal to $$M(Y)$$. Since $$M(Y)$$ means, "The result of applying $$M$$ to the value of $$Y$$," and since $$Y$$ is a measurable function $$\Omega \to \mathbf{R}$$, $$M(Y)$$ must be the measurable function $$\omega \mapsto Y(\omega) \mapsto M(Y(\omega)) = Q_{Y(\omega)}$$, that is, $$M \circ Y$$; so we conclude that $$X = M \circ Y$$ a.s.

I think I understand the problem now: What do we mean by $$X \sim M(Y)$$? I interpreted it one way, and you interpreted it another.

In my interpretation, we sample from $$M(Y)$$ as follows. We sample $$\omega \in \Omega$$; this determines $$\lambda = Y(\omega)$$; and $$M(Y)$$ means $$M(Y(\omega)) = M(\lambda)$$, which is a distribution. To say $$X \sim M(Y)$$ means that the random variable $$X$$ is the result of applying $$M$$ to the random variable $$Y$$, and hence $$X$$ is a random measure.

In your interpretation, we sample from $$M(Y)$$ by sampling $$\omega \in \Omega$$; this determines $$\lambda = Y(\omega)$$; then we sample from $$M(Y(\omega)) = M(\lambda)$$. To say $$X \sim M(Y)$$ means that the distribution of $$X$$ is the distribution $$M(Y)$$ determined by $$Y$$, and hence $$X$$ is a number.

Just for clarity, suppose that instead of $$M$$, we had a function $$f \colon \mathbf{R} \to \mathbf{R}$$. In my interpretation, $$X \sim f(Y)$$ means that the distribution of $$X$$ is the result of sampling $$Y$$ and applying $$Y$$; in yours, $$X \sim f(Y)$$ means that we first apply $$f$$ to the distribution of $$Y$$, and the distribution of $$X$$ is a sample from the result. For example, suppose $$f(x) = x^2$$. I was reading $$X \sim Y^2$$ as, "to sample $$X$$, take a sample from $$Y$$ and square it." You were reading it as, "there is a distribution of $$Y$$'s, and it determines a distribution of squares; to sample $$X$$, take a sample from that distribution of squares." For real-valued $$f$$, it's the same either way. But $$M$$ is measure-valued, and then there's a difference.

I'm not a professional probabilist or statistician, so maybe my interpretation is unorthodox. I think that for your interpretation, I would rather write something like, "$$Y \sim P$$, $$F = M(Y)$$, and $$X \sim F$$." But words are probably clearer.

Having settled that, I can answer the question you asked. I'll be a little pedantic, just to avoid more confusion. I hope you don't mind.

For every $$\lambda \in \mathbf{R}$$, we have a measure $$Q_\lambda$$. We define $$X_\lambda$$ to be a random variable with this law. A priori, it may have some potentially strange event space, but after pushing forward, we may replace the event space with $$\mathbf{R}$$. Then $$X_\lambda$$ is the function from the probability space $$(\mathbf{R}, Q_\lambda)$$ to $$\mathbf{R}$$ defined by $$\lambda \mapsto \lambda$$. The problem statement assumes that $$\mathbf{E}[X_\lambda] = \lambda$$.

We would like to construct an $$X$$ for which $$X \sim M(Y)$$. We want a $$M(Y)$$-distributed sample to be realized by first sampling a value $$Y(\omega)$$, then sampling a value from the random variable $$X_{Y(\omega)}$$. Whatever the event space of $$X$$ is, it must have enough room for us to specify an $$\omega \in \Omega$$, and once that's specified, it must have enough room for the event space of $$X_{Y(\omega)}$$. This could happen, but a priori, there is no reason why it must happen. All we know is that $$\Omega$$ is large enough to support the events we need for $$Y$$. It might happen that $$\Omega = \mathbf{R}$$ and that $$Y$$ is the identity function; in that case, once you have specified $$Y(\omega)$$, you have used up all your randomness. If we tried to use this $$\Omega$$ as the event space of $$X$$, then every $$X_{Y(\omega)}$$ would be deterministic, and that's clearly not what we want.

We can work around this by enlarging the event space. I'll take the event space of $$X$$ to be $$\Omega \times \mathbf{R}$$. I'll endow this space with the product $$\sigma$$-algebra, which I'll denote $$\mathcal{G}$$. I'll give it a measure $$\mu$$ which disintegrates as $$P$$ on $$\Omega$$ and as $$Q_\lambda$$ on $$\{\lambda\} \times \mathbf{R}$$. $$X$$ itself will be the function $$\Omega \times \mathbf{R} \to \mathbf{R}$$ given by $$(\omega, \lambda) \mapsto \lambda$$. And while $$Y$$ is given to us as a random variable with event space $$\Omega$$, we may pull it back to $$\tilde Y$$ on $$\Omega \times \mathbf{R}$$ by simply projecting away the $$\mathbf{R}$$ coordinate: $$\tilde Y(\omega, \lambda) = Y(\omega)$$. The $$\sigma$$-algebra $$\mathcal{F}$$ on $$\Omega$$ pulls back to $$\tilde{\mathcal{F}}$$ on $$\Omega \times \mathbf{R}$$, and $$\sigma(\tilde Y) \subseteq \tilde{\mathcal{F}} \subseteq \mathcal{G}$$, where $$\sigma(\tilde Y)$$ is the $$\sigma$$-algebra generated by $$\tilde Y$$.

Actually, after typing this all out, I notice that your clarification says that $$\Omega$$ "can accommodate all the following random variables." So I guess you knew this already. Sorry.

(And just to connect with what I wrote before: It's the disintegration of $$\mu$$ that defines the random measures I was discussing.)

Now we can make sense of $$\mathbf{E}[X|\tilde Y]$$. (The way I've defined the event spaces, $$\mathbf{E}[X|Y]$$ does not make literal sense because $$X$$ and $$Y$$ are defined on different event spaces, but it's clear that $$\tilde Y$$ and $$Y$$ are effectively equivalent.) By definition, $$\mathbf{E}[X|\tilde Y]$$ is $$\mathbf{E}[X|\sigma(\tilde Y)]$$, and by definition, $$\mathbf{E}[X|\sigma(\tilde Y)]$$ is a $$\sigma(\tilde Y)$$-measurable function $$\Omega \times \mathbf{R} \to \mathbf{R}$$ such that $$\int_A \mathbf{E}[X|\sigma(\tilde Y)] d\mu = \int_A X\,d\mu$$ for every $$A \in \sigma(\tilde Y)$$.

One way to get information about this conditional expectation is to use the tower property to see that $$\mathbf{E}[X|\sigma(\tilde Y)] = \mathbf{E}[\mathbf{E}[X|\tilde{\mathcal{F}}]|\sigma(\tilde Y)]$$. So let's consider $$\mathbf{E}[X|\tilde{\mathcal{F}}]$$. If $$A \in \tilde{\mathcal{F}}$$, then $$A$$ is of the form $$B \times \mathbf{R}$$ for some $$B \in \mathcal{F}$$. Consequently, $$\int_A X\,d\mu = \int_B \left(\int_{\mathbf{R}} X(\omega, \lambda)\,dQ_{Y(\omega)}(\lambda)\right)\,dP(\omega).$$ From the definition of $$X$$, this equals $$\int_B\left(\int_{\mathbf{R}} X_{Y(\omega)}(\lambda)\,dQ_{Y(\omega)}(\lambda)\right)\,dP(\omega) = \int_B \mathbf{E}[X_{Y(\omega)}]\,dP(\omega).$$ Finally, we can apply the assumption that $$\mathbf{E}(X_\lambda) = \lambda$$ to get $$\int_B Y(\omega)\,dP(\omega) = \mathbf{E}_P[Y \cdot 1_B],$$ where $$1_B$$ is the indicator function of $$B$$.

From this, we may deduce that $$\tilde Y$$ is a representative of $$\mathbf{E}[X|\tilde{\mathcal{F}}]$$. (This is analogous to when I wrote $$\mathbf{E}_{\Omega}[X] = Y$$ in my original post, though that was a different $$\Omega$$.) To finish, we plug this into the tower property, and we get the result you wanted: $$\mathbf{E}[X|\tilde Y] = \mathbf{E}[X|\sigma(\tilde Y)] = \mathbf{E}[\tilde Y|\sigma(\tilde Y)] = \tilde Y.$$

• I added some clarification that rigorously defines the more confusing objects involved. I'd also like to point out that I did not state $X = M(Y)$. Commented Nov 11, 2022 at 23:27
• Regarding your edit: There is no issue if $P(Y=y)=0$ since I am never conditioning on this event. I am aware that this issue would exist, if I did say something like $P(X \in A | Y = y)$, which is why I stated it using a bit of measure theory. If we assumed that $P(Y=y) > 0$ then my question would be a lot more trivial. I'm not sure how you got to the conclusion that $X\sim M(Y)$ means that $X$ is now a random measure, but as I defined above, it is not. It is a random variable $\Omega\to\mathbb R$. Commented Nov 12, 2022 at 8:41
• Regarding your second edit: First of all, thank you for the effort. I will upvote since you solved a special case of my question. As I mentioned in my last comment, if I could fix $Y$ ($P(Y=y) > 0$), then it would be simple because I can apply expectations subsequently. Your rephrasing of my question makes the problem a little easier and you use this in the proof. So it doesn't answer my question because you make stronger assumptions. It is a priori not clear that you can actually split $P$ into 2 probability measures, one for $Y$ and one for $X$. Commented Nov 12, 2022 at 22:07