What are the possible measures of uncertainty for a discrete variable X=(x1, x2, ... xn), where values are defined by the alphabet - xi ∈ A, given probabilities p(xi) = P(X = xi) change over time?

E.g. consider time interval T∈( t0, tn) where t0 < t1 <... < tn
1) t = t0 < t1: P = { p(x0), p(x1), ..., p(x2) }
2) t = t1 < t2: P = { p(x0)', p(x1)', ..., p(x2)' }
3) ...

Where p(xi) ≠ p(xi)' and Δt = ti - ti-1 → 0

  • $\begingroup$ A more careful formulation of the question is required: if $X$ is a random variable, the expression $x\in X$ is mysterious. $\endgroup$ – Did Jul 19 '11 at 17:14
  • $\begingroup$ I imagine that the best you can do is to give a measure of uncertainty for a fixed time, eg the variance for given $p(x_i)$. I suppose you could integrate these expressions across time, though it's not quite clear what that would mean. $\endgroup$ – Chris Taylor Jul 19 '11 at 17:37
  • $\begingroup$ @Didier I tried to update the question to make it a bit more clear $\endgroup$ – oleksii Jul 19 '11 at 17:45
  • $\begingroup$ My comment still applies: a random variable is not the set of the values it takes. $\endgroup$ – Did Jul 19 '11 at 17:58
  • $\begingroup$ Like @Chris said (but considering also entropy, apart from variance, as a standard measure of uncertainty). $\endgroup$ – Did Jul 19 '11 at 18:06

Recall that a random variable X is not its image set A but a (measurable) function from a probability space (Ω,F,P) to a (measurable) set A. In your context, A={x1, x2, ... xn} and the distribution of the random variable X is characterized by a collection of nonnegative real numbers p(x1), p(x2), ..., and p(xn) summing to 1.

As @Chris said, your first task is to define a measure of uncertainty, suitable in your context. In other words, for every distribution p one must define a number U(p).

Two options are to use the variance

U(p)=M2(p)-M(p)2 with M2(p)=∑i xi2 p(xi) and M(p)=∑i xi p(xi),

or the entropy

U(p)=−∑i p(xi) log p(xi).

Both quantities are nonnegative, and zero only in the degenerate case when p(xi)=1 for a given i and p(xj)=0 for every other j.

Once the functional U is defined, if the distribution p changes with time t, one might consider the integral

0T U(p(t)) dt

of the functional U applied to p(t) on the time interval of interest.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.