# higher order uncertainty

Context: As a general rule of thumb parameter estimation (in data analysis) works better the fewer the biased assumptions one makes. For instance, when presented with a coin and trying to estimate the true likelihood $H$ for the coin to fall on heads one can first guess a value for $H$ and adapt it as one proceeds with flips. One may claim that the guess $H=0.5$ is the least biased of all choices. I will call this $0$-order uncertainty. Another possibility is not to choose a value for $H$ but rather choose a distribution over the values of $H$. It would seem that choosing the uniform distribution over $[0,1]$ is the least biased one. I will refer to this as first order uncertainty.

It seems natural to adopt the same technique and go on: Instead of choosing a distribution $D\in D([0,1])$, the space of distributions over $[0,1]$, choose instead a distribution $D\in D(D([0,1]))$. Call this second order uncertainty. It would seem that the uniform distribution over $D([0,1])$ expresses a higher degree of uncertainty than the uniform distribution on $[0,1]$ does.

Of course this doesn't stop there. Define $D^n=D(D^{n-1})$, with $D^0=[0,1]$. Choosing an element of $D^n$ for the analysis I will call $n$-order uncertainty. So, the questions are:

1) Is there an example where parameter estimation is done in this higher order context and compared to more ordinary first order uncertainty?

2) Is there a way to formalize the intuitive notion that $n$ order uncertainty expresses less certainty than $n-1$ order uncertainty? Strictly more?

3) Can this idea be pushed to $\infty$? That is, obtain $D^\infty$ as a limit of the $D^n$'s such that $\infty$-order uncertainty then represents the least possible bias.

-
What is the uniform distribution over $D([0,1])$? –  joriki Dec 25 '12 at 22:00
well, the question is very much open for interpretation so I don't know if there is a distribution that can be considered to be uniform on D([0,1]). –  Ittay Weiss Dec 25 '12 at 22:13