I'd like to ask for advice tackling a new (to me) problem in conditional distributions.

Let $X$ and $Y$ be independent real-valued random vectors. Define $Z = H(X) + Y$ where $H$ is a nonlinear, possibly non-invertable map. I want to compute $f_X(x|Z=z^*)$: the posterior distribution of $X$ given the datum $Z = z^*$ (that is, I can measure only $Z$, not $X$ or $Y$, and my measurement of $Z$ is $z^*$) while assuming a prior distribution $f_X(x)$ of $X$ and a known distribution $f_Y(y)$ of $Y$.

My attempt is as follows:

From Bayes' theorem we have

$$f_{X}(x|Z=z^*) = \frac{f_{Z}(Z=z^*|X=x)f_X(x)}{\int f_Z(Z=z^*|X=x')f_X(x')dx'},$$

So we really need $f_{Z}(Z=z^*|X=x)$. I interpreted this as the distribution of $Z$ assuming $X$ is distributed according to $f_X(x)$, which is then evaluated at the specific point $z^*$ (I know there is some subtlety about sets of measure zero here and I may have to take a limit, but I'm not sure of the best way to do this).

Suppose I can determine the distribution of the variable $\phi = H(X)$ from knowledge of $f_X(x)$ (which I know is possible using the Jacobian and inverse map in the case where $H$ is 1-1, for instance). Let's label the distribution of $\phi$ if $X\sim f_X(x)$ by $\pi(\phi)$--then, since $X$ and $Y$, thus $\phi$ and $Y$, are independent, Theorem 7.1 of this PDF implies $f_Z(z)$ is a convolution of $\pi$ and $f_Y$. I get

$$ f_{Z}(Z=z^*|X=x) = \int \pi(z^*-y)f_Y(y)dy $$

for the needed conditional distribution and thus

$$ f_{X}(x|Z=z^*) = \frac{f_X(x)\int \pi(z^*-y)f_Y(y)dy}{\int \int \pi(z^*-y)f_Y(y)f_X(x')dx'dy}, $$

for the final result.

My questions are as follows:

  1. Have I interpreted the conditional $f(Z|X=x)$ correctly? Is it just the distribution of $Z(X,Y)$ assuming $X\sim f_X(x)$? (I am concerned it is more complicated than this).

  2. Is my intuition of evaluating $f_Z(z|X=x)$ at $z^*$ as the way to account for the fact that the measurement of $Z$ was $z^*$ correct? (Same concern as above).

  3. Is there a general formula for the distribution of $\phi$ given $X \sim f_X(x)$ even when $H$ is not invertible? What about the case where $H$ is simply a linear mapping: $H(X) = Hx$?


Theorem 7.1 concerns the distribution of a sum of two random (discrete) variables. That is not what you are looking for, which is the conditional distribution of the sum when given one of the variables.

Since $H()$ is fully determined for any given value of $X$, and as $X,Y$ are independent, thus:

$$f_{Z\mid X}(z\mid x) {= f_{H(X)+Y\mid X}(z\mid x)\\ = f_{Y\mid X}(z-H(x)\mid x)\\ = f_{Y}(z-H(x))} \\~\\ f_{Z,X}(z,x) = f_Y(z-H(x))\; f_X(x)$$

Notice the last line is the joint distribution of $Z$ and $X$.   Obtaining the marginal for $Z$ from this gives the convolution.

So therefore, using this.

$$f_{X\mid Z}(x\mid z^*)~{=\dfrac{f_{Z\mid X}(z^*\mid x) f_X(x)}{\int_\Bbb R f_{Z\mid X}(z^*\mid s) f_X(s)\;\mathrm d s} \\[4ex] = \dfrac{f_Y(z^*-H(x))\;f_X(x)}{\int_\Bbb R f_Y(z^*-H(s))\;f_X(s)\operatorname d s } }$$

  • $\begingroup$ Thanks! Don't know how I missed that! $\endgroup$ – SZN Aug 21 '17 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.