I have a large set of numbers and these numbers are being pretty much grouped into clusters. For example let's say i have $5,000$-$6,000$ numbers between $8$ and $45$, a $2,000$-$3,000$ set in the $200$ to $250$ range, and finally a set of $400$-$500$ numbers in the $2500$-$3000$ range.

What I need to do I distribute more evenly the values so I have bigger distinction between a value at $8$ and value at $45$. For example. Let say the minimum value is $8$ and max is $3000$ in the whole set. Typically the ratio value of $1500$ should be about $50$% while a $750$ should get typically $25$%. The problem is there is not enough distinction within the cluster. What I would like to know is if there is a method or formula that exists to distribute such clusters to the next cluster so that I can the whole range covered. Like the first group would increase its values to fill in numbers between $45$ and the next group of $200$.

I might be complicated to understand so here a small example of what i am trying to get from a specific result. I do not say what i want is the "exact" output values i want for that set but it's just to explain the principle i am looking for

Input : $[1,2,3,4,5,6,7,8,9,300,301,302,2000,2001,2002]$

Output :$[1,45,75,100,155,185,200,235,270,300,600,850,1200,1700,2002]$

I am trying to spread without perfect spreading; I have to keep it tight enough around each clusters.

  • $\begingroup$ How would you use the more spread-out results? $\endgroup$
    – Paul
    Commented Feb 10, 2020 at 19:02
  • $\begingroup$ I think one idea is: suppose you have a list of $n$ numbers, with $a$ being the smallest number and $b$ being the largest number. Uniformly sample $n$ numbers $\sim U[a,b]$ and this gives you a new list of $n$ numbers without perfect spreading. There are further things you could do like set up clusters $\cup_{i=1}^k [a_i, b_i]$. Then decide how many numbers you want in each cluster and sample from these intervals. $\endgroup$
    – fGDu94
    Commented Feb 10, 2020 at 19:17
  • $\begingroup$ @Paul the data points are not perfectly realistic values, Having the realistic values take many days of computing. Smoothing the value by hand was done for couple hundred tests so far and it come pretty darn close to the reality. Still by hand it take couple hours. $\endgroup$
    – Franck
    Commented Feb 10, 2020 at 19:36
  • $\begingroup$ @fGDu94 Yes i thought of that, simply sorting them out and using a basic value equal to the index but that is too linear, i need to figure out a bell curve / sine wave curve between the data point clusters. $\endgroup$
    – Franck
    Commented Feb 10, 2020 at 19:38
  • $\begingroup$ you could sample from a Continuous univariate distribution supported on a bounded interval, e.g. Beta, arcsine, logit-normal. These are all supported on the interval $[0,1]$. I.e. $x_i \in [0,1]$. Then do the transformation $(b-a)x_i + a \in [a,b]$ to map it onto the interval you want. $\endgroup$
    – fGDu94
    Commented Feb 10, 2020 at 20:19


You must log in to answer this question.

Browse other questions tagged .