# Possible Mistake in Calculating Posterior of Distribution using Bayes Rule and Integration

I have been struggling on a homework question where I have to compute the posterior density of a distribution. While I can compute the posterior, I believe I made a mistake because the area under the posterior density tends to infinity.

I have spent hours double checking my work and have had several friends also come up with the same answer, so I was hoping that someone could take a look at the work and either spot the mistake or explain what is going on.

The setup is as follows. Let:

• $y = (y_1,\ldots,y_N)$
• $p_x(x) = \exp(-x)$
• $p_{y_i|x} = xy_i^{x-1} = x\exp((x-1)\log{y_i})$ for $y_i \in (0,1)$ and $x>0$

Given this information, we can calculate $p_{x|y}(x|y)$ using Bayes' Rule as follows:

\begin{align} p_{x|y}(x|y) &\propto p_{y|x}(y|x)p_x(x) \\ &\propto \prod_{i=1}^N p_{y_i|x}(y_i|x)p_x(x)\\ &\propto \prod_{i=1}^N x\exp((x-1)\log{y_i})\exp(-x)\\ &\propto x^N\exp((x-1)\sum_{i=1}^N\log{y_i})\exp(-Nx)\\ &\propto x^N\exp(x(\sum_{i=1}^N\log{y_i}-N))\exp(\sum_{i=1}^N\log{y_i})\\ \end{align}

Letting $S=\sum_{i=1}^N\log{y_i}$, this results in:

\begin{align} p_{x|y}(x|y) = \frac{1}{Z}x^N\exp(x(S-N)\exp(-S) \end{align}

Where $Z$ denotes a normalization constant that can be calculated as:

\begin{align} Z &= \int_{x>0}p_{y|x}(y|x)p_x(x)dx \\ &= \int_{x>0}x^N\exp(x(S-N))\exp(-S)dx \\ &= \exp(-S)\frac{1}{N-S}\int_{x>0}x^N(N-S)\exp(-x(N-S))\exp(-S)dx \\ &= \exp(-S)\frac{1}{N-S}\mathbb{E}[X^N] \\ &= \exp(-S)\frac{1}{N-S}\frac{N!}{(N-S)^N}\\ &= \exp(-S)\frac{N!}{(N-S)^{N+1}}\\ \end{align}

Note that I have evaluated the above integral using the $N$th moment of an exponential random variable.

Putting everything together, this gives me:

\begin{align} p_{x|y}(x|y) &= \frac{x^N\exp(x(S-N))\exp(-S)}{\exp(-S)\frac{N!}{(N-S)^{N+1}}}\\ &= \frac{(N-S)^{N+1}}{N!} x^N\exp(x(S-N))\\ \end{align}

Unfortunately, this density approaches infinity as $x \rightarrow \infty$, and so it does not normalize to 1.

• "this density approaches infinity as x→∞"... how do you arrive at that conclusion? – Glen_b Apr 24 '13 at 5:00

$$\int_0^\infty p(x|y)dx=\int_0^\infty (S-1)^2x e^{(-x(S-1))}dx=(S-1)\int_0^\infty (S-1)x e^{(-x(S-1))}dx=(S-1)E(V)$$ where $V\sim Exp\left(\frac{1}{(S-1)}\right)$. The expected value of an exponentially distributed variable is its parameter. Hence,
$$\int_0^\infty p(x|y)dx=(S-1)E(V)=\frac{(S-1)}{(S-1)}=1$$
• Hmm, I used a negative sign in front of $x$ in my calculations (which you apparently didn't have), but that should not change anything. Since $y_i\in(0,1)$ the sum will be negative. Since the parameter in an exponential must be positive, you should probably define $S$ as $-\sum_{i=1}^n\log y_i$ instead of without the negative sign. And then you get a negative sign in front of the $x$ as I then used. – hejseb Apr 24 '13 at 6:00