4
$\begingroup$

The excercise from the book

I am solving problem 1.4 of the famous book Pattern recognition and machine learning of Bishop. The idea of the exercise is that in a simple function $f(x)$ the maximum is the same if we apply a transformation $x = g(y)$. However, in the case of a probability density, this does not hold anymore. I have solved the exercise, but Bishop says this happens because of the Jacobian factor and I do not understand what this means.

Could someone help me with this concept?

$\endgroup$

1 Answer 1

3
$\begingroup$

Welcome to MSE! Conceptually the idea is the following: if $f$ is a probability density function, it satisfies certain properties, like $f \geq 0$ and $$\int f(x) d x = 1.$$ If we look at a transformation $f(g(y))$, firstly the second property might be not true anymore. Hence, $f(g(\cdot))$ is likely no probability density function. Secondly, what we roughly try to describe with $f(g(\cdot))$ is the probability distribution of a random variable $Y$ that is given such that $g(Y)$ follows the distribution represented by $f$. Say $g$ is invertible and sufficiently smooth. The distribution of $Y$ is given by $$P(Y \in A) = P(g(Y) \in g(A)) = \int_{g(A)} f(x) dx.$$ Which is not very useful in practice, as this integral is on sets of the form $g(A)$. According to the integration by substitution formula, we can compute a probability density function $h$ such that $$P(Y \in A) = \int_A h(y)dy.$$ Here, $$h = f(g(\cdot))\det J_g(\cdot),$$ where $J_g$ is the Jacobian of $g$. (Bishop gives this formula in (1.27)).

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .