# Joint distribution of a Dirichlet with repeating components

Let $$(X_1, \dots, X_k) \sim Dir(\alpha_1, \dots, \alpha_k)$$. If we let $$Z_1 = X_1+X_2$$, what would the joint distribution of $$X_1, Z_1$$ be? Same with the posterior distribution of $$X_1$$ given $$Z_1$$?

I know that the distribution of $$Z_1$$ is $$Dir(\alpha_1+\alpha_2, \sum_{i=3}^{k} \alpha_i)$$ and that the distribution of $$X_1$$ is $$Beta(\alpha_1, \sum_{i=2}^{k} \alpha_i)$$, but I don't know how to combine the two to get their joint distribution.

Based on the properties I've found, I'd have guessed something like $$(X_1, Z_1, 1-X_1-Z_1) \sim Dir(\alpha_1, \alpha_1+\alpha_2, 1-2\alpha_1-\alpha_2$$), but the fact that the $$\alpha_1$$ is repeating doesn't seem right to me

• What have you tried? Where is it that you are stucked? Jan 24, 2019 at 5:32
• I was just having trouble with the fact that $X_1$ is repeating. Do we subtract $\alpha_1$ from the sum of $\alpha$'s twice? Jan 24, 2019 at 6:43

The distribution of $$(X_1, X_2)$$ is given by $$\text{Dir}(\alpha_1, \alpha_2, \sum_{i = 3}^{n} \alpha_i)$$ (proof below). Therefore the joint density of $$(X_1, X_2)$$ is $$f_{X_1, X_2}(x_1, x_2) = \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}x_1^{\alpha_1 - 1}x_2^{\alpha_2 - 1}(1 - x_1 - x_2)^{(\sum_{i = 3}^n \alpha_i) - 1},$$ where $$x_1, x_2 > 0$$ and $$x_1 + x_2 < 1$$.

Now we do a transformation of variables: Let $$Z = X_1 + X_2$$ and $$X = X_1$$ (I'm using $$Z, X$$ instead of $$Z_1, X_1$$ because, later on, the subscripts look messy otherwise). Then $$f_{X, Z}(x, z) = f_{X_1, X_2}(x_1(x, z), x_2(x, z))\left|\det\left(\frac{d(x_1, x_2)}{d(x, z)} \right) \right|.$$ The Jacobian here is $$1$$ and so the joint density is $$f_{X, Z}(x, z) = \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}x^{\alpha_1 - 1}(z - x)^{\alpha_2 - 1}(1 - z)^{(\sum_{i = 3}^n \alpha_i) - 1},$$ where $$x > 0$$, $$z > x$$ and $$z < 1$$.

The marginal density of $$Z$$ is $$\text{Dir}(\alpha_1 + \alpha_2, \sum_{i = 3}^{n} \alpha_i)$$, so the conditional density of $$X$$ given $$Z$$ is $$f_{X \mid Z}(x \mid z) = \frac{f_{X, Z}(x, z)}{f_{Z}(z)} = \frac{\frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}x^{\alpha_1 - 1}(z - x_1)^{\alpha_2 - 1}(1 - z)^{(\sum_{i = 3}^n \alpha_i) - 1}}{\frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1} + \alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}z^{\alpha_1 + \alpha_2 - 1}(1 - z)^{(\sum_{i = 3}^n \alpha_i) - 1}},$$ which simplifies to $$f_{X \mid Z}(x \mid z) = \frac{1}{z}\frac{\Gamma(\alpha_1 + \alpha_2)}{\Gamma(\alpha_1)\Gamma(\alpha_2)}\left(\frac{x}{z}\right)^{\alpha_1 - 1}\left(1 - \frac{x}{z}\right)^{\alpha_2 - 1}.$$

Proof that the distribution of $$(X_1, X_2)$$ is given by $$\text{Dir}(\alpha_1, \alpha_2, \sum_{i = 3}^{n} \alpha_i)$$:

Let $$I(x, k, n) = \{(x_k, \dots, x_n): x_k, \dots, x_n > 0, \sum_{i = k}^n x_i = x\}$$. We know that $$X_1 \sim \text{Beta}(\alpha_1, \sum_{i = 2}^{n} \alpha_i)$$. Therefore $$\int_{I(1 - x_1, 2, n)} f_{X_1, \dots, X_n}(x_1, \dots, x_n) dx_2\cdots dx_n = \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\sum_{i = 2}^{n}\alpha_i)}x_1^{\alpha_1 - 1}(1 - x_1)^{(\sum_{i = 2}^n \alpha_i) - 1}.$$ Hence $$\int_{I(1 - x_1, 2, n)} x_2^{\alpha_2 - 1}\cdots x_n^{\alpha_n - 1} dx_2\cdots dx_n = \frac{\Gamma(\alpha_2)\cdots\Gamma(\alpha_n)}{\Gamma(\sum_{i = 2}^n \alpha_i)}(1 - x_1)^{(\sum_{i = 2}^n \alpha_i) - 1}.$$ Therefore $$f_{X_1, X_2}(x_1, x_2) = \frac{\Gamma(\sum_{i = 1}^n \alpha_i)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_n)}x_1^{\alpha_1 - 1}x_2^{\alpha_2 - 1} \int_{I(1 - x_1 - x_2, 3, n)} x_3^{\alpha_3 - 1}\cdots x_n^{\alpha_n - 1} dx_2\cdots dx_n$$ $$= \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_2)\Gamma(\sum_{i = 3}^{n}\alpha_i)}x_1^{\alpha_1 - 1}x_2^{\alpha_2 - 1}(1 - x_1 - x_2)^{(\sum_{i = 3}^n \alpha_i) - 1}$$

• Thanks so much for your response! Jan 25, 2019 at 19:52