0
$\begingroup$

I am creating an objective function for an algorithm. I have an array that I need to return a single value from (and minimise in the optimisation). For the moment I only use the mean of the array. However, I would also like to consider the variance of that array (and minimise it as well).

Either I use the variance and the mean as independent variables to minimise (not ideal in my opinion), or combine the two values into a single one. Is there a method to combine the mean and the variance of an array into a single value? I was going to simply do: $$ \frac{Var_{norm}+Mean_{norm}}{2} $$

where $Var_{norm}$ is the variance normalised and $Mean_{norm}$ is the mean if the array normalised. I have a feeling that there must be something better out there. I have been looking but cannot find anythin.

Any ideas?

Thank you

$\endgroup$
7
  • $\begingroup$ Well, may be you can use point estimates or a regression model! $\endgroup$ Mar 9, 2020 at 14:27
  • 2
    $\begingroup$ The end result is that since you are using the values for whatever you are doing with them, that you can choose for yourself what to do with the values. If you want to average the mean and the variance... go ahead. If you want to put more importance on one rather than the other, consider multiplying it by a larger constant... Just so long as you know what the value means and can explain it to someone who asks what it means, it sounds like what you are doing with them is entirely personal preference. $\endgroup$
    – JMoravitz
    Mar 9, 2020 at 14:28
  • $\begingroup$ @JMoravitz is correct. In general, you could do something like $\mathbb{E}[X] + \lambda \operatorname{Var}(X)$ where $X$ is your random variable and $\lambda$ is a positive number. Right now, you've effectively chosen $\lambda = 1$. $\endgroup$
    – parsiad
    Mar 9, 2020 at 14:29
  • $\begingroup$ define a complex number where the real part is the mean and the imaginary part the variance $\endgroup$ Mar 9, 2020 at 14:31
  • $\begingroup$ For analogy, if you were to ask someone playing a game like Dungeons and Dragons what the best weapon is, you'll get varying answers based on that player's preference... maybe they like fist weapons since you get several attacks meaning more consistency, maybe they like rapiers since they crit more often allowing you to ignore your opponent's armor but deal less average damage, maybe they like heavy axes since their top-end damage is higher but miss more often, ... you will have many different opinions on whether average damage, high end damage, or consistency is important with no right answer $\endgroup$
    – JMoravitz
    Mar 9, 2020 at 14:35

3 Answers 3

3
$\begingroup$

I don't think there is an accepted combination of those two statistics that makes theoretical sense.

I suggest you consider a weighted average $$ t \times \text{ mean } + (1-t) \times \text{ variance} $$ and experiment with various values of $t$ between $0$ and $1$ to see what works in your particular application.

If the mean and variance have substantially different orders of magnitude the ordinary average ($t= 1/2$) will skew toward the larger of the two.

$\endgroup$
3
  • $\begingroup$ I'm just reading up on weighted average on wikipedia and it seems exactly like this is exactly what I need! Thanks a million $\endgroup$
    – Melclic
    Mar 9, 2020 at 14:48
  • $\begingroup$ Do use the standard deviation, as @RossMillikan suggests in his answer. $\endgroup$ Mar 9, 2020 at 18:19
  • $\begingroup$ I haven't implemented it just yet, but yes it seems that a weighted average with standard deviation is the way to go $\endgroup$
    – Melclic
    Mar 9, 2020 at 18:59
3
$\begingroup$

The units of variance are the square of the units of the mean, so it does not make sense to add them. If the units of the mean are length, the units of the variance are length$^2$, so the minimization will depend on whether you measure in centimeters or meters. You can solve this by using the standard deviation instead of the variance.

If you use $\frac {\text{mean + standard deviation}}2$ you will minimize the mean $+1$ standard deviation point. You can weight them to get other points in the distribution. Maybe you want to minimize mean $+3$ sigma with the sense that you will minimize the "worst case". You would then use $\frac {\text{mean + 3*standard deviation}}4$

$\endgroup$
2
$\begingroup$

I agree with remarks that suggest that it depends on the purpose of what you are doing.

In evolutionary biology, fitness is often defined as the expectation of the number of offspring for a trait. When there are two competing traits--e.g. because there is a gene at which there are two alleles (alternative versions of the gene)--then in a large population, over the short term, and with some other assumptions about the two distributions over numbers of offspring, it's likely that the fitter trait will increase in relative frequency compared to the less fit trait. So this definition of fitness provides a useful estimate indicating what is likely to happen or what has been happening in the population.

In Natural Selection for Variances in Offspring Numbers: A New Evolutionary Principle and in other papers, John Gillespie pointed out that variance in number of offspring for organisms with a given, heritable trait can affect which trait is likely to increase in frequency over the long term.

When there are environmental variations that affect all members of a population at the same time, geometric mean number of offspring is a better predictor than the expectation. As a simple estimate of this effect, Gillespie uses

$$\mu - \frac{\sigma^2}{2}$$

where $\mu$ is the expectation of number of offspring, and $\sigma^2$ is the variance.

When there is per-organism variation in probabilities of numbers of offspring for organisms with the same trait, Gillespie uses the estimate

$$\mu - \frac{\sigma^2}{N}$$

where $N$ is the size of the population. That is, in this situation, calculating this value for each of the competing traits provides a better estimate than the expectation $\mu$ of which trait is likely to increase in frequency relative to the other one.

The point of describing Gillespie's results is not to argue that you should use either of these functions to combine the expectation and variance, but just to show that the best way two combine the two values depends on the application.

An alternative would be to leave the expectation and variance as distinct quantities, and think of them as determining a two-dimensional landscape. You want to minimize both, I gather, so the task is to find the low points on this landscape. That's more complicated than minimizing a single-valued function of expectation and variance. There might be more local minima that are not global minima, or it may be that you end up with a saddle shape in which you can't minimize both quantities, but if you figured this out, you would have more understanding of the situation.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .