3
$\begingroup$

I understand that the expectation of a random variable $X_i$ a Dirichlet distribution is $E[X_i] = \frac{\alpha_i}{\sum_k \alpha_k}$ and $E[\ln(X_i)] = \psi(\alpha_i) - \psi(\sum_k \alpha_k)$

I read a great paper "Distribution of Mutual Information from Complete and Incomplete Data" that states that $E[X_i\ln(X_i)] = \sum_{i} \frac{\alpha_i}{\sum_k \alpha_k} \{\psi(\alpha_i) - \psi(\sum_k \alpha_k)\}$

I was wondering what is $E[X_i X_j]$?

Can I use the property $\operatorname{Cov}(X_i X_j) = E[X_i X_j] - E[X_i] E[X_j]$ to get $E[X_i X_j] = \operatorname{Cov}(X_i X_j) + E[X_i] E[X_j]$?

What about $E[X_i \ln(X_j)]$? Can I also use the $\operatorname{Cov}$ to derive it?

Latest update:

I believe that $E[X_i \ln(X_j)]$ can be derived using $\operatorname{Cov}[X_i, \ln(X_j)]$

I found that

$\mathrm{Cov}[X_i,X_j] = \frac{- \alpha_i \alpha_j}{\alpha^2 (\alpha+1)} \ \ \ $ where $\alpha = \sum_{i=1}^K\alpha_i$

and

$\operatorname{Cov}[\log(X_i),\log(X_j)] = \psi'(\alpha_i) \delta_{ij} - \psi'(\alpha_0)$ where $\psi$ is the digamma function, $\psi'$ is the trigamma function, and $\delta_{ij}$ is the Kronecker delta

Is there a form for $\operatorname{Cov}[X_i,\log(X_j)]$? If so, we can use it to derive $E[X_i \ln(X_j)]$.

$\endgroup$
  • $\begingroup$ I am confused. What are you asking? $\endgroup$ – Lost1 Jan 23 '14 at 17:51
  • $\begingroup$ @Lost1 I apologise for the poor English, I have amended the question. My main question is how to derive $E[X_i \ln(X_j)]$ $\endgroup$ – Michael Jan 24 '14 at 2:34
2
$\begingroup$

$$E(X_i)=\frac{\alpha_i}{\alpha}\qquad E(X_iX_j)=\frac{\alpha_i\alpha_j}{\alpha(\alpha+1)}\qquad\alpha=\sum_k\alpha_k$$ Edit: Recall that for every $i\ne j$ the density $f$ of the distribution $f$ of $(X_i,X_j)$ is such that $$ f(x,y)=\frac{\Gamma(\alpha)}{\Gamma(\alpha_i)\Gamma(\alpha_j)\Gamma(\alpha-\alpha_i-\alpha_j)}x^{\alpha_i-1}y^{\alpha_j-1}(1-x-y)^{\alpha-\alpha_i-\alpha_j-1}\mathbf 1_D(x,y), $$ where $$D=\{(x,y)\mid x\gt0,y\gt0,x+y\lt1\}.$$ Hence, $$ E(X_i\log X_j)=\iint_Dx\log y\,f(x,y)\,\mathrm dx\mathrm dy, $$ that is, $$ E(X_i\log X_j)=\frac{\Gamma(\alpha)}{\Gamma(\alpha_i)\Gamma(\alpha_j)\Gamma(\alpha-\alpha_i-\alpha_j)}\iint_Dx^{\alpha_i}y^{\alpha_j-1}(1-x-y)^{\alpha-\alpha_i-\alpha_j-1}\log y\,\mathrm dx\mathrm dy. $$ The change of variable $x=(1-y)z$ yields $$ E(X_i\log X_j)=\frac{\alpha_i}\alpha\frac{\Gamma(\alpha+1)}{\Gamma(\alpha_j)\Gamma(\alpha-\alpha_j+1)}\int_0^1y^{\alpha_j-1}(1-y)^{\alpha-\alpha_j}\log y\,\mathrm dx\mathrm dy=\frac{\alpha_i}\alpha E(\ln Y), $$ where $Y$ is beta $(\alpha_j,\alpha-\alpha_j+1)$. According to the WP page on the beta distribution, $$ E(\ln Y)=\psi(\alpha_j)-\psi(\alpha+1), $$ hence $$ E(X_i\log X_j)=\frac{\alpha_i}\alpha(\psi(\alpha_j)-\psi(\alpha+1)). $$

$\endgroup$
  • $\begingroup$ Thank you very much for your answer. Do you have the form for $E[X_i \ln(X_j)]$? $\endgroup$ – Michael Jan 24 '14 at 2:46
  • $\begingroup$ As an integral on $[0,1]\times[0,1]$, sure. $\endgroup$ – Did Jan 24 '14 at 10:49
  • $\begingroup$ can you show me how? Thanks! $\endgroup$ – Michael Jan 24 '14 at 11:10
  • $\begingroup$ Whoa! Thank you so much! You are a genius. Do you have any recommendations for a good book on Dirichlet distribution? I could not find any references that talk about joint Dirichlet distribution. $\endgroup$ – Michael Jan 24 '14 at 14:16
  • $\begingroup$ @Michael Your edit is incorrect. Users Dan and Eupraxis1981: Before "Approving" an Edit, please make sure it is not absurd. To believe that $α_i=α−α_j$ and/or $αΓ(α)=Γ(α+1)$ could be "change of variables" is a sure sign one does not know what is going on. $\endgroup$ – Did Jan 24 '14 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.