Let $\Omega$ be a set of $n$ elements.

I need to do the following two things:

  1. Describe the set $\mathcal{P}$ of all probability distributions supported on $\Omega$.
  2. Show that $\mathcal{P}$ is convex.

Now, the second part is probably very easy once I've figured out the first part. For the first part, I was given the hint: "represent a distribution on $\Omega$ by a vector and describe its properties to obtain an analytic description of $\mathcal{P}$".

What I'm confused about is, are we talking about cumulative distribution functions? Are we talking about probability density functions? Because $\Omega$ consists of $n$ elements, are we talking about discrete probability distributions? Or do we just mean that for any given distribution, the probabilities of elements are between $0$ and $1$ and add up to (or integrate to, if it's continuous) to $1$?

The wording of Part 1 is very confusing to me and I don't understand how to represent a distribution on $\Omega$ by a vector. Also, what does it mean that the probability distribution is "supported" on $\Omega$? Usually, the support of a function is the set of all $x$ such that the function value is nonzero, so does this mean that no point in $\Omega$ gives us a zero probability?

I apologize for my absolute cluelessness, but I really need help understanding this! Could somebody please help me?

Thank you.

  • $\begingroup$ For starters, how does your textbook or teacher define "probability distribution" on a finite set $\Omega$? $\endgroup$ – kimchi lover Sep 28 '17 at 21:50
  • $\begingroup$ @kimchilover they don't because it's not a probabilty course - it's a nonlinear optimization course. When I asked for clarification though, our prof said "that $\Omega$ as a sample space contains finitely many simple events to which we assign probabilities. Thus, any distribution is completely described by stating what are the probabilities of those simple events. " $\endgroup$ – ALannister Sep 28 '17 at 21:53
  • $\begingroup$ @kimchilover this is what I'm thinking: Let $X$ be probability distribution supported on $\Omega$. Then, for $\{\omega_{1}, \omega_{2}, \cdots , \omega_{n}\} \in \Omega$, let $x:= (p(\omega_{1}),p(\omega_{2}) , \cdots , p(\omega_{n})) \in \mathbb{R}^{n}$. Then, since $x$ is supported on $\Omega$, $p(\omega_{i})>0$ $\forall i$, and since $\sum_{i=1}^{n} p(\omega_{i})=1$, $p(\omega_{i})<1$ $\forall i$. Is that enough of an analytic description? $\endgroup$ – ALannister Sep 28 '17 at 22:06
  • $\begingroup$ Sounds good to me. You have reduced the question to one about a set of vectors. I might also consider the case where $p(\omega_i)=0$ is allowed, and see if the answers are different in that case, too. $\endgroup$ – kimchi lover Sep 28 '17 at 22:26
  • $\begingroup$ hint $\endgroup$ – Nadiels Sep 28 '17 at 22:50

WLOG assume that $\Omega = \{1,2,\ldots,n\}$. Let $\mathcal F=2^\Omega$. A probability distribution $P\in\mathcal P$ is a map $P:\mathcal F\to\mathbb R$ such that

  • $P(E)\geqslant0$ for $E\in\mathcal F$.
  • $P(A\cup B) = P(A)+P(B)$ for $A,B\in\mathcal F$ with $A\cap B=\varnothing$.
  • $P(\Omega)=1$.

The set $\mathcal P$ then is just the collection of measures on $\Omega$ which assign the value $1$ to $\Omega$. Now let $P,Q\in\mathcal P$ and $\lambda\in(0,1)$. It is clear that $\lambda P(E) + (1-\lambda)Q(E)\geqslant 0$ for each $E\in\mathcal F$ and that $\lambda P(\Omega) + (1-\lambda)Q(\Omega) = 1$. If $A,B\in\mathcal F$ are disjoint, then $$ \lambda P(A\cup B) + (1-\lambda)Q(A\cup B) = \lambda(P(A)+P(B)) + (1-\lambda)(Q(A)+Q(B)), $$ and so $\lambda P + (1-\lambda)Q\in\mathcal P$. It follows that $\mathcal P$ is convex.

  • $\begingroup$ is this set also closed? $\endgroup$ – ALannister Oct 23 '17 at 0:07
  • $\begingroup$ Closed in what sense? There are various topologies one may consider on a set of probability distributions. $\endgroup$ – Math1000 Oct 25 '17 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.