# How to measure the amount of uncertainty

What are the possible measures of uncertainty for a discrete variable X=(x1, x2, ... xn), where values are defined by the alphabet - xi ∈ A, given probabilities p(xi) = P(X = xi) change over time?

E.g. consider time interval T∈( t0, tn) where t0 < t1 <... < tn
1) t = t0 < t1: P = { p(x0), p(x1), ..., p(x2) }
2) t = t1 < t2: P = { p(x0)', p(x1)', ..., p(x2)' }
3) ...

Where p(xi) ≠ p(xi)' and Δt = ti - ti-1 → 0

• A more careful formulation of the question is required: if $X$ is a random variable, the expression $x\in X$ is mysterious. – Did Jul 19 '11 at 17:14
• I imagine that the best you can do is to give a measure of uncertainty for a fixed time, eg the variance for given $p(x_i)$. I suppose you could integrate these expressions across time, though it's not quite clear what that would mean. – Chris Taylor Jul 19 '11 at 17:37
• @Didier I tried to update the question to make it a bit more clear – oleksii Jul 19 '11 at 17:45
• My comment still applies: a random variable is not the set of the values it takes. – Did Jul 19 '11 at 17:58
• Like @Chris said (but considering also entropy, apart from variance, as a standard measure of uncertainty). – Did Jul 19 '11 at 18:06

Recall that a random variable X is not its image set A but a (measurable) function from a probability space (Ω,F,P) to a (measurable) set A. In your context, A={x1, x2, ... xn} and the distribution of the random variable X is characterized by a collection of nonnegative real numbers p(x1), p(x2), ..., and p(xn) summing to 1.

As @Chris said, your first task is to define a measure of uncertainty, suitable in your context. In other words, for every distribution p one must define a number U(p).

Two options are to use the variance

U(p)=M2(p)-M(p)2 with M2(p)=∑i xi2 p(xi) and M(p)=∑i xi p(xi),

U(p)=−∑i p(xi) log p(xi).

Both quantities are nonnegative, and zero only in the degenerate case when p(xi)=1 for a given i and p(xj)=0 for every other j.

Once the functional U is defined, if the distribution p changes with time t, one might consider the integral

0T U(p(t)) dt

of the functional U applied to p(t) on the time interval of interest.