1
$\begingroup$

I am trying to construct groups that share common interests. I have sets of values representing interest in a topic (they could be any subjective measure, like availability during a particular segment of a schedule, etc). I want a single number that represents how well each group shares an interest in a topic.

The edge cases are easy as are any homogenous sets. For example,

f([0.0, 0.0, 0.0]) = 0.0
f([1.0, 1.0, 1.0]) = 1.0
f([0.5, 0.5, 0.5]) = 0.5

My first impulse was to use mean, but it doesn't take into account that if there is a wide variance, the score should be lower. It represent that one of the group members is less interested in the topic (or less available).

mean([1.0, 1.0, 0.0]) = 0.67

This hypothetical function should have a downward bias, as disinterest by one member pulls the whole score down. Or put another way, a group member's low value can bring down a cluster of high numbers more dramatically than a single group member's high value can raise a cluster of low values.

I desperately tried a few things before resorting to asking the experts.

f(array) = average(array)-stdev(array)
f(array) = average(array)-(stdev(array)/2)

All of these felt pretty clumsy.

Is there a statistic or common mathematical operation that is used to measure shared interest?

$\endgroup$
1
$\begingroup$

How about the geometric mean:

$$\sqrt[n]{x_1 x_2\cdots x_n}$$

The geometric mean of $[x, x, \dots, x]$ is $x$. Meanwhile, the AM–GM inequality says that the geometric mean of a collection of nonnegative numbers is always less than or equal to their arithmetic mean, with equality if and only if all the numbers in the list are the same. This sounds like the behavior you want to me.

Also, if any $x_i$ is zero, then the whole result is zero, which seems like a plausible behavior for a function computing the degree of overlap of availability or "interest values".

Another option is the harmonic mean. The harmonic mean is defined as the reciprocal of the arithmetic mean of the reciprocals of the inputs:

$$\frac{n}{\frac{1}{x_1}+\dots+\frac{1}{x_n}}$$

In practice, you want to add a special case to set the output to zero if any of the inputs is zero, since that makes the behavior continuous at zero. The harmonic mean of a collection of nonnegative numbers is always less than or equal to their geometric mean. In other words, it gives even more extreme "reduction due to small values" behavior than the geometric mean.

You could also just take the minimum of your input values: $\min(1.0, 0.3, 0.5) = 0.3$. This gives the most extreme "reduction due to small values" behavior that still makes sense.

The generalized mean includes all of the above as special cases. It gives you a parameter $p$ that allows you to smoothly vary from the minimum function through the harmonic mean, geometric mean, and arithmetic mean depending on what value you set $p$ to.


Edit: In a comment, OP asked whether there is a function that behaves similarly but does not output zero if only one of the inputs is zero. You could get behavior like this by taking (for example) the 25th percentile of the distribution of input values. See this question for info about predicting percentiles from small sample sets. Also, while it doesn't directly answer the question, everyone working on ranking systems should read Evan Miller's article "How Not To Sort By Average Rating" at some point.

That said, it's hard to see how such a function would make sense as an answer to the problem as stated, since you are essentially saying that it is OK to put someone in a group for which they have 0% availability (or 0% professed interest in being involved).

Here are some ways I can think of to interpret this problem that still make sense:

  1. People can exaggerate their (dis)interest in being in a group. This is like how YouTube stopped using the five-star rating system because everyone only gave one-star and five-star ratings. In this case, Bayesian models like the ones from Evan Miller's articles might be appropriate.

  2. Everyone has to be put into some group, even if this results in some people being in groups for which they have zero interest. At this point you have something like a weighted constraint satisfaction problem and everything gets way more difficult.

$\endgroup$
  • $\begingroup$ This is closer. I note that f(1.0, 1.0, 0.1) = 0.46 which seems high considering one person's "interest" is almost non-existent. $\endgroup$ – Wes Modes Feb 19 '18 at 16:19
  • $\begingroup$ @Wes Did you try the harmonic mean or minimum function (two of the other special cases of the generalized mean)? $\endgroup$ – Aaron Rotenberg Feb 19 '18 at 19:19
  • $\begingroup$ I wasn't as clear how to implement the harmonic mean from looking at the wiki page. I didn't see the minimum function. Do you have a pointer to that? Does it approach zero with out zeroing out? $\endgroup$ – Wes Modes Feb 21 '18 at 16:59
  • 1
    $\begingroup$ @WesModes Added more info on harmonic mean and minimum function. $\endgroup$ – Aaron Rotenberg Feb 22 '18 at 15:22
  • $\begingroup$ Geometric mean and harmonic mean all return zero if any of the inputs are zero. Is there any value of p in the generalized mean (or another central tendency computation) that approach but do not return zero in this case? $\endgroup$ – Wes Modes Feb 25 '18 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.