Suppose I am collecting a set of samples from a finite-support 1D distribution (e.g. a top hat function of some finite width around 0). The distribution is continuous. If the sample size is finite, say n, I expect the largest sample for different repetitions of my measurement to fluctuate according to some distribution. This largest-sample-distribution will have some finite variance V(n).

Now suppose that I increase the sample size n. This should produce a decrease in V(n), as I now have a higher probability for the largest sample to fall within some arbitrary dx from the distribution's upper bound.

First question: how does V(n) fall with increasing n? I would expect it to tend to zero in the n-to-infinity limit. However, not in the same way it would do so for a discrete distribution, due to the different 'orders of infinity' (cardinalities) characterising sample size and sample-space size.

Second question: how does the above compare to the case of an infinite-support distribution, say a 1D Gaussian around 0? Does V(n) still tend to zero in the n-to-infinity limit? In terms of cardinality of the sample-space size there should be no difference, but is it so?


For a uniform distribution on $[0,1]$, the maximum of a sample size $n$ has a Beta distribution with parameters $\alpha=n$ and $\beta=1$, an expectation of $\dfrac{n}{n+1}$ and a variance of $\dfrac{n}{(n+1)^2(n+2)}$ which as a function of $n$ decreases towards zero as $n$ increases, in line with what you seem to expect. If the range of the uniform distribution is $k$ rather than $1$ then this multiplies the variance by $k^2$ but you get the same reducing effect as the sample size increases.

You are correct that for any distribution with finite support, the variance of the sample maximum will eventually tend towards zero as the sample size increases without limit.

For a distribution with an infinite support (at least to the right), you should expect the relationship between the sample size and the variance of the sample maximum to depend on how heavy the right-hand tail of the distribution is. You can find a lot more information than you probably need by reading about Extreme Value Theory.

For a standard Gaussian distribution, the variance of the sample maximum seems to tend towards zero as sample size increases, but much more slowly than with a uniform distribution (roughly proportionate to $\frac{1}{\log n}$ rather than to $\frac{1}{n^2}$). For an exponential distribution with rate parameter $1$, the variance of the sample maximum seems to increase as sample size increases, to a finite limit of $\frac{\pi^2}{6}$ as suggested by the Gumbel distribution. For a Pareto power law distribution (with parameter greater than $2$) the variance of the sample maximum seems to increase without limit as sample size increases.

| cite | improve this answer | |
  • $\begingroup$ Thank you. Does this mean that, unless we know the exact form of the original distribution right tail, there is no way of distinguishing between finite and infinite-support distributions, given that we (empirically) observe V(n) to decrease with n? $\endgroup$ – Luluca Aug 9 '16 at 9:35
  • $\begingroup$ Finite-support distributions will see faster decreases in the variance of the sample maximum than most infinite-support distributions. But if you wanted to judge whether a distribution had finite or infinite support, I suspect you might do better to look at the pattern of the running maximum as the samples grow rather than looking at the variance. $\endgroup$ – Henry Aug 9 '16 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.