Approximately normal. Suppose the data are something like normal, with a concentration of values
near the middle (mean or median) with 'tails' of straggling values roughly
symmetrically in both direction, then a general solution is easy.
In that case, if you want 40% of the observations, you could find the
the 30th and 70th percentiles, and the interval between them would be about the
shortest that contains 40% of the observations.
Specifically, consider a sample of size 500 sampled from a normal population with mean 100
and standard deviation 15 (rounded to tenths). Here is a histogram of such a sample, with tick marks below showing specific locations of the observations. (The software I used is R, other software would work similarly.)
x = round(rnorm(500, 100, 15),1) # generate fake data as described
hist(x, prob=T, col="wheat"); rug(x) # make figure
quantile(x, c(.30, .70)) # find 30th and 70th percentiles
abline(c(91.84, 108.63), col="red") # red lines in figure
sum(x > 91.84 & x < 108.63) # verify exactly 200 in interval
The red lines show the interval $(91.84, 108.63)$ from the 30th to the 70th percentile. I have verified that it does indeed contain 40% of the observations: $.40(500) = 200.$
I would not want to claim that this is absolutely the very shortest interval
that contains 200 of the 500 observations, but it is not far off. Most of the
observations are concentrated around 100.
Exponential, By contrast, if the population is right-skewed with a sparse tail out to the
right, then the greatest concentration will be near the low end. So, roughly speaking, the shortest interval containing 40% of the observations would be
between the minimum and thee 40th percentile.
Specifically, consider a sample of size 500 sampled from an exponential population with mean 100 (rate 1/100) and rounded to 100ths. Now the interval is
very nearly $(0, 46.446)$.
x = round(rexp(500, .01),2)
hist(x, prob=T, col="wheat")
sum(x < 46.44)
It is possible that by chance there might be a cluster of several tied
observations near 46.44 or 46.45, and you would have to choose whether
to include 199, 200, or 201 observations.
General. Finally, if you insist on absolutely the shortest interval or you have no idea
about the shape of the distribution, then you could look at many pairs
of quantiles that include 40% of the data: '.0 to .40', '.1 to .41', and so
on to '.60 to 1.0'. Find the length of each of these, and pick the shortest.
It sounds tedious, but a simple computer program could do it in a flash.