1
$\begingroup$

Given a set of real numbers, and a wanted percentage x, what is the most efficient way to get the minimal sized range that contains $x\%$ of the numbers in the set?

Edit : can Confidence Interval be usefull on a set with unknown distribution? https://en.wikipedia.org/wiki/Confidence_interval

Edit 2 :

size of range is (upper bound - lower bound), by demending that it will be minimized, it is easy to see that upper bound and lower bound must be taken from the set itself.

to clearify even more, i need to find x such that at least x/100*(size of the set) numbers from the set are within upper bound and lower bound of the range that is to find.

$\endgroup$
  • $\begingroup$ What do you know about the set? All the values? The maximum and minimum? The size of the set? $\endgroup$ – Henry Sep 1 '16 at 18:33
  • $\begingroup$ I have all items in the set, real numbers for instance. But i dont know them in advance, I get them "out of the blue" and I have to find that range then. Cant assume anything about their distribution although I will eventually assume its normal if no other solution found... $\endgroup$ – Ofek Ron Sep 1 '16 at 18:37
  • $\begingroup$ There a number of important considerations here: What do you mean by size of range? What do you mean by $x$% of real numbers? I know the latter comes down to what the probability measure on $\mathbb{R}$ is, but you seem to be implying that is an unknown here. The notion of size of the range is ambiguous to me, maybe you mean cardinality? But there are infinite subsets of $\mathbb{R}$ that you might consider smaller than others even though their cardinalities are the same (e.g. interval $(0,1)$ is same size(cardinality) as entirety of reals). $\endgroup$ – Justin Benfield Sep 1 '16 at 18:45
  • 1
    $\begingroup$ @JustinBenfield you went too far... i clearified my question in Edit 2, please tell me if there is anything missing still $\endgroup$ – Ofek Ron Sep 1 '16 at 18:54
  • $\begingroup$ There is a formalism, not so complicated, quickly learnable... The wiki : Sample size determination . $\endgroup$ – user354674 Sep 2 '16 at 0:03
1
$\begingroup$

Approximately normal. Suppose the data are something like normal, with a concentration of values near the middle (mean or median) with 'tails' of straggling values roughly symmetrically in both direction, then a general solution is easy.

In that case, if you want 40% of the observations, you could find the the 30th and 70th percentiles, and the interval between them would be about the shortest that contains 40% of the observations.

Specifically, consider a sample of size 500 sampled from a normal population with mean 100 and standard deviation 15 (rounded to tenths). Here is a histogram of such a sample, with tick marks below showing specific locations of the observations. (The software I used is R, other software would work similarly.)

x = round(rnorm(500, 100, 15),1)         # generate fake data as described
hist(x, prob=T, col="wheat");  rug(x)    # make figure
quantile(x, c(.30, .70))                 # find 30th and 70th percentiles
   30%    70% 
 91.84 108.63 
abline(c(91.84, 108.63), col="red")      # red lines in figure
sum(x > 91.84 & x < 108.63)              # verify exactly 200 in interval
## 200

enter image description here

The red lines show the interval $(91.84, 108.63)$ from the 30th to the 70th percentile. I have verified that it does indeed contain 40% of the observations: $.40(500) = 200.$

I would not want to claim that this is absolutely the very shortest interval that contains 200 of the 500 observations, but it is not far off. Most of the observations are concentrated around 100.

Exponential, By contrast, if the population is right-skewed with a sparse tail out to the right, then the greatest concentration will be near the low end. So, roughly speaking, the shortest interval containing 40% of the observations would be between the minimum and thee 40th percentile.

Specifically, consider a sample of size 500 sampled from an exponential population with mean 100 (rate 1/100) and rounded to 100ths. Now the interval is very nearly $(0, 46.446)$.

 x = round(rexp(500, .01),2)
 hist(x, prob=T, col="wheat")
 rug(x)
 quantile(x, .40)
    40% 
 46.446 
 sum(x < 46.44)
 ## 200

enter image description here

It is possible that by chance there might be a cluster of several tied observations near 46.44 or 46.45, and you would have to choose whether to include 199, 200, or 201 observations.

General. Finally, if you insist on absolutely the shortest interval or you have no idea about the shape of the distribution, then you could look at many pairs of quantiles that include 40% of the data: '.0 to .40', '.1 to .41', and so on to '.60 to 1.0'. Find the length of each of these, and pick the shortest. It sounds tedious, but a simple computer program could do it in a flash.

$\endgroup$
  • $\begingroup$ The General approach you suggest seem to fit my needs, thanks! $\endgroup$ – Ofek Ron Sep 4 '16 at 9:43
  • $\begingroup$ There are other approaches, depending on what percentage 'x' you want to include. Especially, if 'x' is rather low on the one hand or around 90% or 95% on the other. I just chose 40% for a general example Please edit 'Addendum' to your Question if you want to be more specific or to explain your purpose. $\endgroup$ – BruceET Sep 4 '16 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.