Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

This is a question on how to make sense of an equation (specifically, a description of a distribution) that, no matter how I slice it and dice it, always has some left over pieces that don't fit.

The equation in question describes a random variable $G$ drawn from a Dirichlet process $\textrm{DP}(\alpha_0,G_0)$: \begin{align} G = \sum_{k=1}^\infty \beta_k \delta_{\theta_k} \end{align} where $\alpha_0 > 0$ and $G_0$ is some base measure. The $\beta_k$ are stick breaking weights that depend on $\alpha_0$ and $\theta_k$ are atoms drawn from $G_0$.

The full description can be found at the top of page 4 of this paper.

My first question is whether I can read $G$ like a function. Say, if I have some "atom" (which term I don't fully understand) $\theta_j$, can I plug it in like: \begin{align} G(\theta_j) &= \sum_{k=1}^\infty \beta_k \delta_{\theta_k} (\theta_j) \newline &= \sum_{k \:\textrm{where} \: \theta_k = \theta_j} \beta_k \end{align} meaning if I happened to draw an atom that was identical to more than one $\theta_k$ I am just adding up the relevant weights $\beta_k$. And that the random variable $G$ itself can be loosely understood as very long vector of weights $(\ldots,G_j = \sum_{k \:\textrm{where} \: \theta_k = \theta_j} \beta_k,\ldots)$ such that it is just a repartitioning of the infinite dimension vector $(\ldots,\beta_k,\ldots)$.

But if I interpret it this way, I get into trouble because apparently, you're supposed to use the $G$ to draw the so-called "atoms" again. So I revise my understanding to think of $G$ as a multinomial distribution. But then the equation that sums over the infinitely many values of $\beta_k$ doesn't make any sense. A sum is a sum. How do you draw things from a sum, which has just a fixed scalar value?

Is my interpretation wrong? Whether it's wrong at the foundations or wrong in subtle ways, please correct me regardless.

I'm trying to make the leap from the prob/statistics in DeGroot's book to understanding measure theory and nonparametric processes and it's things like these that completely throw me for a loop.

share|improve this question
1  
Think of a discrete probability measure, then each point that has positive measure is an atom. For example, a Bernoulli distribution has atoms 0 and 1. Then, $G$ can be seen as a random discrete probability measure with atoms at $\theta_k$, with (random) mass $\beta_k$. To draw a sample according to $G$, it means to draw a point $\theta_k$ according to the weight $\beta_k$. (In general, DP processes can be formulated as point processes. But this might be too much for the time being.) –  Morning Feb 22 '11 at 1:08
    
@Morning Thanks! The Bernoulli analogy definitely clears up some things. I can generalize straightforwardly from there to a case where there are $n$ distinct values with $n$ distinct $\beta$s. Two questions, if you don't mind. (1) Can the atoms be any scalar (including complex numbers) or even any complex-valued vector? (2) What if the base distribution $G_0$ is finite/discrete valued (e.g. a Bernoulli)? Is it still valid under this formalism to sample ones and zeros infinitely many times, add up the corresponding $\beta$s and say that is the mass for corresponding $G$? –  JasonMond Feb 22 '11 at 2:22
1  
@JasonMond: (1) I think the points $\theta_k$ can take values in even more abstract metric spaces (hence including the complex numbers and vectors), according your choice of $G_0$. (2) $G_0$ can well be a Bernoulli distribution. In this case, you can sample all the $\theta_k$'s as a process (infinitely many zeros and ones), or, I would add the $\beta_k$'s and see the obtained $G$ as a new (mixed, if $\beta_k$'s are random) Bernoulli distribution. –  Morning Feb 22 '11 at 2:55
    
@Morning Thanks on (1)! I'm a little confused about the two cases you mention in (2). I don't understand why they are two different cases. Just to keep this concrete, let's say the infinite $\beta$ vector is defined $(1/2,1/4,1/8,\ldots,0.5^k,\ldots)$ and the $\theta$ vector is $(0,1,0,1,\ldots)$ with every even atom 1 and odd atom 0. Then isn't the ensemble for $G$ just $P(G=0)=2/3$ and $P(G=1)=1/3$? This would be the second case you mention. But how would this be different from your first case? Or is there a way to do this without generating $\beta$? –  JasonMond Feb 22 '11 at 3:38
    
@JasonMond: Sorry that I was misleading and confusing. I think your interpretation was correct. I was thinking of a slightly different type of processes... :P –  Morning Feb 22 '11 at 13:19
show 1 more comment

1 Answer

up vote 1 down vote accepted

For a discrete probability measure, each point that has positive measure is an atom. For example, a Bernoulli distribution has atoms 0 and 1. Then, $G$ can be seen as a random discrete probability measure with atoms at $\theta_k$, with (random) mass $\beta_k$. To draw a sample according to $G$, it means to draw a point $\theta_k$ according to the weight $\beta_k$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.