Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

1. If $X_1,X_2,\ldots,X_n$ are discrete r.v.'s with joint pmf $f(x_1,\ldots,x_n|\theta)$. Let theta be a discrete random variable with prior pmf $\pi(\theta)$. Let $H(x_1,x_2,\ldots,x_n)$ be a sufficient statistic. Show that $\pi(\theta|x_1=x_1,\ldots,x_n=x_n) = \pi(\theta|H=h)$.

Attempt: Basically the posterior distribution of theta given the sample is equivalent to the posterior of theta given a sufficient statistic of the sample as a sufficient statistic contains all the relevant information about the sample.

share|cite|improve this question

I'm going to write $X$ for $(X_1,\ldots,X_n)$ and $x$ for $(x_1,\ldots,x_n)$.

The likelihood function when $X$ is observed to have a certain value $x$ is $$ L_1(\theta) = \Pr(X=x \mid \theta) = E(\Pr(X=x\mid H(X),\theta)) = \sum_h \Pr(H(X)=h\mid\theta)\Pr(X=x\mid H(X)=h). $$ In the very last probability, we don't need to write $\Pr(X=x\mid H(X)=h,\theta)$, since lack of this dependence on $\theta$ is the very definition of sufficiency. The index $h$ of course runs through the discrete set of all possible values of the random variable $H(X)$.

Now $\Pr(X=x\mid H(X)=h) = 0$ for any value of $h$ except $h=H(x)$. So all but one of the terms in the last sum vanish and the sum is $$\Pr(H(X)=h\mid\theta)\Pr(X=x\mid H(X)=h) =\Pr(H(X)=H(x)\mid\theta)\Pr(X=x\mid H(X)=H(x)).$$ The second factor in the last product does not depend on $\theta$. So as a function of $\theta$, it's constant. Therefore the likelihood function when $H(X)$ is observed to have a certain value $H(x)$ is $$ L_2(\theta) = \Pr(H(X) = h\mid \theta) = \mathrm{constant}\cdot \Pr(X=x\mid \theta) = \mathrm{constant} \cdot L_1(\theta). $$

Bayes' theorem says if you multiply the likelihood function by the prior pmf and then normalize, you get the posterior pmf.

The "constant" factor referred to above goes away when you normalize; it appears as a factor in both the numerator and the denominator and cancels. Therefore you get the same posterior pmf if you observe $H(X)$ that you get if you observe $X$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.