4
$\begingroup$

I understand that the expectation of a random variable $X_i$ a Dirichlet distribution is $E[X_i] = \frac{\alpha_i}{\sum_k \alpha_k}$ and $E[\ln(X_i)] = \psi(\alpha_i) - \psi(\sum_k \alpha_k)$

I read a great paper "Distribution of Mutual Information from Complete and Incomplete Data" that states that $E[X_i\ln(X_i)] = \sum_{i} \frac{\alpha_i}{\sum_k \alpha_k} \{\psi(\alpha_i) - \psi(\sum_k \alpha_k)\}$

I was wondering what is $E[X_i X_j]$?

Can I use the property $\operatorname{Cov}(X_i X_j) = E[X_i X_j] - E[X_i] E[X_j]$ to get $E[X_i X_j] = \operatorname{Cov}(X_i X_j) + E[X_i] E[X_j]$?

What about $E[X_i \ln(X_j)]$? Can I also use the $\operatorname{Cov}$ to derive it?

Latest update:

I believe that $E[X_i \ln(X_j)]$ can be derived using $\operatorname{Cov}[X_i, \ln(X_j)]$

I found that

$\mathrm{Cov}[X_i,X_j] = \frac{- \alpha_i \alpha_j}{\alpha^2 (\alpha+1)} \ \ \ $ where $\alpha = \sum_{i=1}^K\alpha_i$

and

$\operatorname{Cov}[\log(X_i),\log(X_j)] = \psi'(\alpha_i) \delta_{ij} - \psi'(\alpha_0)$ where $\psi$ is the digamma function, $\psi'$ is the trigamma function, and $\delta_{ij}$ is the Kronecker delta

Is there a form for $\operatorname{Cov}[X_i,\log(X_j)]$? If so, we can use it to derive $E[X_i \ln(X_j)]$.

$\endgroup$
2
  • $\begingroup$ I am confused. What are you asking? $\endgroup$
    – Lost1
    Commented Jan 23, 2014 at 17:51
  • $\begingroup$ @Lost1 I apologise for the poor English, I have amended the question. My main question is how to derive $E[X_i \ln(X_j)]$ $\endgroup$
    – Michael
    Commented Jan 24, 2014 at 2:34

1 Answer 1

2
$\begingroup$

$$E(X_i)=\frac{\alpha_i}{\alpha}\qquad E(X_iX_j)=\frac{\alpha_i\alpha_j}{\alpha(\alpha+1)}\qquad\alpha=\sum_k\alpha_k$$ Edit: Recall that for every $i\ne j$ the density $f$ of the distribution $f$ of $(X_i,X_j)$ is such that $$ f(x,y)=\frac{\Gamma(\alpha)}{\Gamma(\alpha_i)\Gamma(\alpha_j)\Gamma(\alpha-\alpha_i-\alpha_j)}x^{\alpha_i-1}y^{\alpha_j-1}(1-x-y)^{\alpha-\alpha_i-\alpha_j-1}\mathbf 1_D(x,y), $$ where $$D=\{(x,y)\mid x\gt0,y\gt0,x+y\lt1\}.$$ Hence, $$ E(X_i\log X_j)=\iint_Dx\log y\,f(x,y)\,\mathrm dx\mathrm dy, $$ that is, $$ E(X_i\log X_j)=\frac{\Gamma(\alpha)}{\Gamma(\alpha_i)\Gamma(\alpha_j)\Gamma(\alpha-\alpha_i-\alpha_j)}\iint_Dx^{\alpha_i}y^{\alpha_j-1}(1-x-y)^{\alpha-\alpha_i-\alpha_j-1}\log y\,\mathrm dx\mathrm dy. $$ The change of variable $x=(1-y)z$ yields $$ E(X_i\log X_j)=\frac{\alpha_i}\alpha\frac{\Gamma(\alpha+1)}{\Gamma(\alpha_j)\Gamma(\alpha-\alpha_j+1)}\int_0^1y^{\alpha_j-1}(1-y)^{\alpha-\alpha_j}\log y\,\mathrm dx\mathrm dy=\frac{\alpha_i}\alpha E(\ln Y), $$ where $Y$ is beta $(\alpha_j,\alpha-\alpha_j+1)$. According to the WP page on the beta distribution, $$ E(\ln Y)=\psi(\alpha_j)-\psi(\alpha+1), $$ hence $$ E(X_i\log X_j)=\frac{\alpha_i}\alpha(\psi(\alpha_j)-\psi(\alpha+1)). $$

$\endgroup$
7
  • $\begingroup$ Thank you very much for your answer. Do you have the form for $E[X_i \ln(X_j)]$? $\endgroup$
    – Michael
    Commented Jan 24, 2014 at 2:46
  • $\begingroup$ As an integral on $[0,1]\times[0,1]$, sure. $\endgroup$
    – Did
    Commented Jan 24, 2014 at 10:49
  • $\begingroup$ can you show me how? Thanks! $\endgroup$
    – Michael
    Commented Jan 24, 2014 at 11:10
  • $\begingroup$ Whoa! Thank you so much! You are a genius. Do you have any recommendations for a good book on Dirichlet distribution? I could not find any references that talk about joint Dirichlet distribution. $\endgroup$
    – Michael
    Commented Jan 24, 2014 at 14:16
  • $\begingroup$ @Michael Your edit is incorrect. Users Dan and Eupraxis1981: Before "Approving" an Edit, please make sure it is not absurd. To believe that $α_i=α−α_j$ and/or $αΓ(α)=Γ(α+1)$ could be "change of variables" is a sure sign one does not know what is going on. $\endgroup$
    – Did
    Commented Jan 24, 2014 at 17:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .