# Set of all probability distributions supported on $\Omega$ is a convex set.

Let $\Omega$ be a set of $n$ elements.

I need to do the following two things:

1. Describe the set $\mathcal{P}$ of all probability distributions supported on $\Omega$.
2. Show that $\mathcal{P}$ is convex.

Now, the second part is probably very easy once I've figured out the first part. For the first part, I was given the hint: "represent a distribution on $\Omega$ by a vector and describe its properties to obtain an analytic description of $\mathcal{P}$".

What I'm confused about is, are we talking about cumulative distribution functions? Are we talking about probability density functions? Because $\Omega$ consists of $n$ elements, are we talking about discrete probability distributions? Or do we just mean that for any given distribution, the probabilities of elements are between $0$ and $1$ and add up to (or integrate to, if it's continuous) to $1$?

The wording of Part 1 is very confusing to me and I don't understand how to represent a distribution on $\Omega$ by a vector. Also, what does it mean that the probability distribution is "supported" on $\Omega$? Usually, the support of a function is the set of all $x$ such that the function value is nonzero, so does this mean that no point in $\Omega$ gives us a zero probability?

I apologize for my absolute cluelessness, but I really need help understanding this! Could somebody please help me?

Thank you.

• For starters, how does your textbook or teacher define "probability distribution" on a finite set $\Omega$? – kimchi lover Sep 28 '17 at 21:50
• @kimchilover they don't because it's not a probabilty course - it's a nonlinear optimization course. When I asked for clarification though, our prof said "that $\Omega$ as a sample space contains finitely many simple events to which we assign probabilities. Thus, any distribution is completely described by stating what are the probabilities of those simple events. " – ALannister Sep 28 '17 at 21:53
• @kimchilover this is what I'm thinking: Let $X$ be probability distribution supported on $\Omega$. Then, for $\{\omega_{1}, \omega_{2}, \cdots , \omega_{n}\} \in \Omega$, let $x:= (p(\omega_{1}),p(\omega_{2}) , \cdots , p(\omega_{n})) \in \mathbb{R}^{n}$. Then, since $x$ is supported on $\Omega$, $p(\omega_{i})>0$ $\forall i$, and since $\sum_{i=1}^{n} p(\omega_{i})=1$, $p(\omega_{i})<1$ $\forall i$. Is that enough of an analytic description? – ALannister Sep 28 '17 at 22:06
• Sounds good to me. You have reduced the question to one about a set of vectors. I might also consider the case where $p(\omega_i)=0$ is allowed, and see if the answers are different in that case, too. – kimchi lover Sep 28 '17 at 22:26
• hint – Nadiels Sep 28 '17 at 22:50

WLOG assume that $\Omega = \{1,2,\ldots,n\}$. Let $\mathcal F=2^\Omega$. A probability distribution $P\in\mathcal P$ is a map $P:\mathcal F\to\mathbb R$ such that
• $P(E)\geqslant0$ for $E\in\mathcal F$.
• $P(A\cup B) = P(A)+P(B)$ for $A,B\in\mathcal F$ with $A\cap B=\varnothing$.
• $P(\Omega)=1$.
The set $\mathcal P$ then is just the collection of measures on $\Omega$ which assign the value $1$ to $\Omega$. Now let $P,Q\in\mathcal P$ and $\lambda\in(0,1)$. It is clear that $\lambda P(E) + (1-\lambda)Q(E)\geqslant 0$ for each $E\in\mathcal F$ and that $\lambda P(\Omega) + (1-\lambda)Q(\Omega) = 1$. If $A,B\in\mathcal F$ are disjoint, then $$\lambda P(A\cup B) + (1-\lambda)Q(A\cup B) = \lambda(P(A)+P(B)) + (1-\lambda)(Q(A)+Q(B)),$$ and so $\lambda P + (1-\lambda)Q\in\mathcal P$. It follows that $\mathcal P$ is convex.