3
$\begingroup$

(Wikipedia page for background info: https://en.wikipedia.org/wiki/Principle_of_transformation_groups )


I'm reading Edwin Jaynes's paper, "Prior Probabilities":

http://bayes.wustl.edu/etj/articles/prior.pdf

and on page 17 he describes the principle of transformation groups, wherein he provides a method to derive objective prior distributions for different parameters of a given probability density function.

Trouble is, I don't get how it works.

Jaynes seems to be trying to find a change of variables for a pdf such that the new pdf, multiplied by the Jacobian determinant of the new parameterization, results in the exact same pdf as before the transformation. Then, he tries to find a prior distribution for a given set of parameters such that the prior distribution, when run through the same change of variables and multiplied by the same Jacobian determinant, remains the same.

His notation is a little vague, so here's my interpretation of one of his examples. Let's say you have a reparameterization of $\frac{1}{\sigma}f(\frac{x-\mu}{\sigma})dx$, denoted $\Phi$, where $\Phi(x,\mu, \sigma)=(a(x+b),a(\mu+b),a\sigma)$, where $a$ and $b$ are arbitrary values.. Plugging these new variables into $\frac{1}{\sigma}f(\frac{x-\mu}{\sigma})dx$, we have $\frac{1}{a\sigma}f(\frac{a(x+b)-a(\mu+b)}{a\sigma})dx'\mathbf{J_{\Phi}}$, which simplifies to:

$\frac{1}{a\sigma}f(\frac{x-\mu}{\sigma})dx' \left| \left( \begin{array}{ccc} a & 0 & 0 \\ 0 & a & 0 \\ 0 & 0 & a \end{array} \right) \right| = \frac{a^{2}}{\sigma}f(\frac{x-\mu}{\sigma})dx' $

... which isn't the same as the original, though Jaynes says that it is. He then proceeds to take the Jacobian and apply it to a prior, which I also don't understand how he's able to do. Aren't Jacobian determinants supposed to multidimensional versions of the coefficient of $du$ in $u$ substitution? Why is Jaynes able to multiply one seemingly unrelated function by the Jacobian determinant of another?

I'm positive I've done something wrong with my calculations, or made a fumble in my reasoning somewhere. If anyone could run me through how this method works, I'd be very grateful.

$\endgroup$
0
$\begingroup$

He's applying the Jacobian to do a change of variables. When you do a change of variables you distort the total mass assigned to all of the subsets of the support. The Jacobian adjusts the integrand appropriately to provide the same mass over the original domain of integration.

In general, if you have a probability density $f_X(x)$ and you do a change of variables to $x\rightarrow y$ where $y = g(x)$ then $f_Y(y) = |\frac{d}{dx}g^{-1}(x)|f(g^{-1}(x))$.

He uses this, using his notation now, and works under the assumption that $h(x) = h(\frac{x-\mu}{\sigma})$ for some location and scale parameters $\mu$ and $\sigma$. He then just comes up with a clever transformation of variables:

\begin{equation} \begin{split} \mu' &= \mu +b\\ \sigma' &= a\sigma\\ x' &= a(x-\mu) \end{split} \end{equation}

The Jacobian of this transform is $|J|=a^{-1}$. Because: \begin{equation} h(\frac{x-\mu}{\sigma})\frac{1}{\sigma} = h(\frac{\frac{x'-\mu'}{a}}{\frac{\sigma'}{a}})\frac{a}{\sigma'}|J| = h(\frac{\frac{x'-\mu'}{a}}{\frac{\sigma'}{a}})\frac{aa^{-1}}{\sigma'} \end{equation}

all of the $a$'s cancel out and we get the original equation in terms of the new variables.

Doing the change of variables on a prior distribution $f(\mu,\sigma)$ gives us $f(\mu,\sigma)\rightarrow af(\mu',\sigma') = af(\mu+b,a\sigma) $, which he calls $g(\mu',\sigma')$.

Under our assumption, shifts and scales of the parameters do not change the prior probabilities, so we need to find $f(\mu,\sigma)$ such that $f(\mu,\sigma) = af(\mu+b,a\sigma)$ almost surely with respect to Lebesgue measure.

This is satisfied by $f(\mu,\sigma) \propto \frac{1}{\sigma}$, which can be easily checked by plugging in $af(\mu+b,a\sigma) = \frac{a}{a\sigma} = \frac{1}{\sigma}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.