# What is the expectation of the product of two random variables of Dirichlet distribution?

I understand that the expectation of a random variable $X_i$ a Dirichlet distribution is $E[X_i] = \frac{\alpha_i}{\sum_k \alpha_k}$ and $E[\ln(X_i)] = \psi(\alpha_i) - \psi(\sum_k \alpha_k)$

I read a great paper "Distribution of Mutual Information from Complete and Incomplete Data" that states that $E[X_i\ln(X_i)] = \sum_{i} \frac{\alpha_i}{\sum_k \alpha_k} \{\psi(\alpha_i) - \psi(\sum_k \alpha_k)\}$

I was wondering what is $E[X_i X_j]$?

Can I use the property $\operatorname{Cov}(X_i X_j) = E[X_i X_j] - E[X_i] E[X_j]$ to get $E[X_i X_j] = \operatorname{Cov}(X_i X_j) + E[X_i] E[X_j]$?

What about $E[X_i \ln(X_j)]$? Can I also use the $\operatorname{Cov}$ to derive it?

Latest update:

I believe that $E[X_i \ln(X_j)]$ can be derived using $\operatorname{Cov}[X_i, \ln(X_j)]$

I found that

$\mathrm{Cov}[X_i,X_j] = \frac{- \alpha_i \alpha_j}{\alpha^2 (\alpha+1)} \ \ \$ where $\alpha = \sum_{i=1}^K\alpha_i$

and

$\operatorname{Cov}[\log(X_i),\log(X_j)] = \psi'(\alpha_i) \delta_{ij} - \psi'(\alpha_0)$ where $\psi$ is the digamma function, $\psi'$ is the trigamma function, and $\delta_{ij}$ is the Kronecker delta

Is there a form for $\operatorname{Cov}[X_i,\log(X_j)]$? If so, we can use it to derive $E[X_i \ln(X_j)]$.

• I am confused. What are you asking? – Lost1 Jan 23 '14 at 17:51
• @Lost1 I apologise for the poor English, I have amended the question. My main question is how to derive $E[X_i \ln(X_j)]$ – Michael Jan 24 '14 at 2:34

$$E(X_i)=\frac{\alpha_i}{\alpha}\qquad E(X_iX_j)=\frac{\alpha_i\alpha_j}{\alpha(\alpha+1)}\qquad\alpha=\sum_k\alpha_k$$ Edit: Recall that for every $i\ne j$ the density $f$ of the distribution $f$ of $(X_i,X_j)$ is such that $$f(x,y)=\frac{\Gamma(\alpha)}{\Gamma(\alpha_i)\Gamma(\alpha_j)\Gamma(\alpha-\alpha_i-\alpha_j)}x^{\alpha_i-1}y^{\alpha_j-1}(1-x-y)^{\alpha-\alpha_i-\alpha_j-1}\mathbf 1_D(x,y),$$ where $$D=\{(x,y)\mid x\gt0,y\gt0,x+y\lt1\}.$$ Hence, $$E(X_i\log X_j)=\iint_Dx\log y\,f(x,y)\,\mathrm dx\mathrm dy,$$ that is, $$E(X_i\log X_j)=\frac{\Gamma(\alpha)}{\Gamma(\alpha_i)\Gamma(\alpha_j)\Gamma(\alpha-\alpha_i-\alpha_j)}\iint_Dx^{\alpha_i}y^{\alpha_j-1}(1-x-y)^{\alpha-\alpha_i-\alpha_j-1}\log y\,\mathrm dx\mathrm dy.$$ The change of variable $x=(1-y)z$ yields $$E(X_i\log X_j)=\frac{\alpha_i}\alpha\frac{\Gamma(\alpha+1)}{\Gamma(\alpha_j)\Gamma(\alpha-\alpha_j+1)}\int_0^1y^{\alpha_j-1}(1-y)^{\alpha-\alpha_j}\log y\,\mathrm dx\mathrm dy=\frac{\alpha_i}\alpha E(\ln Y),$$ where $Y$ is beta $(\alpha_j,\alpha-\alpha_j+1)$. According to the WP page on the beta distribution, $$E(\ln Y)=\psi(\alpha_j)-\psi(\alpha+1),$$ hence $$E(X_i\log X_j)=\frac{\alpha_i}\alpha(\psi(\alpha_j)-\psi(\alpha+1)).$$
• Thank you very much for your answer. Do you have the form for $E[X_i \ln(X_j)]$? – Michael Jan 24 '14 at 2:46
• As an integral on $[0,1]\times[0,1]$, sure. – Did Jan 24 '14 at 10:49
• @Michael Your edit is incorrect. Users Dan and Eupraxis1981: Before "Approving" an Edit, please make sure it is not absurd. To believe that $α_i=α−α_j$ and/or $αΓ(α)=Γ(α+1)$ could be "change of variables" is a sure sign one does not know what is going on. – Did Jan 24 '14 at 17:59