# Are these three ways of normalizing a set/vector of numbers 'equivalent'?

Say I have a set of positive numbers (well, 0 is allowed, but if 0 is the max then no further operation is done and hence the special case of divide by zero won't arise). These are essentially some ratings - could all be on different scales.

Now I wish to 'normalize' the set so that it's between [0,1] (or [0,10] or [0,100] etc.,) Here are the three ways and I'm really confused about when to choose one over the other or are they equivalent?

Given:

$\qquad S$ = $\{1,3,7,9,11,12,34\}$

Possible ways of normalizing: (I may be incorrect and hence the question)

1. First way: divide eah element by the total sum of all elements in the set:$\begin{equation*} \forall s \in S: s_i^' = \Large\frac{s_i}{\sum_{i=1}^ns_i} \end{equation*} \qquad\small i = 1\space to \space n; \space n = |S| \space and \space s_i^'= is\space normalized \space quantity.;\space Ex: \space \frac{1}{77}, \frac{3}{77},...,\frac{34}{77}$

(In order to scale to a different interval, say [0,10] just multiply $s_i^'$ by 10)

2. Second way: Find the max of the set and divide each element by it

$$$\forall s \in S: s_i^' = \Large\frac{s_i}{max(S)}$$ \space;Ex: \space \frac{1}{34}, \frac{3}{34},...,\frac{34}{34}=1$

Another way: The same as the above, but this time based on interpretation: i.e., if a smaller number is a better 'rating' than a bigger number e.g., a 1 is better than a 9. So in this case the order would just reverse:

$$$\forall s \in S: s_i^' = \Large\frac{min(S)}{s_i}$$\space;Ex: \space \frac{1}{1}=1, \frac{1}{3},\frac{1}{7},...,\frac{1}{34}$

(here too in order to scale to a different interval, say [0,10] just multiply $s_i^'$ by 10)

3. Third way:

$$$\forall s \in S: s_i^' = rating *\large\frac{worst(S) - s_i}{worst(S) - best(S)}$$$

$where:$

$\qquad rating = 1$ if required interval is [0,1]

$\qquad worst(S) = max(S)$ if 'lower' rating is better i.e., 1 is better than a 9

$\qquad\qquad\qquad\qquad = min(S)$ otherwise

$\qquad best(S) = min(S)$ if 'lower' rating is better i.e., 1 is better than a 9

$\qquad\qquad\qquad\space = max(S)$ otherwise (basically the opposite of $worst(S)$)

Example: Assume that a higher rating is better i.e., $best(S)=34 \space and \space worst(S)=1$. Normalization interval: [0,1] thus $rating=1$

$s_1 = 1 * \frac{1-1}{1-34} = 0$ notice this method has a zero value!

$s_2 = 1 * \frac{1 - 2}{1 - 34} = 0.30303$

$s_7 = 1 * \frac{1-34}{1-34} = 1$

So in this technique the worst value is given the score of zero and the best gets the score of 1. Depending on the interval the value of the rating is chosen as the multiplying factor

Context of usage: Assume that each such set of numbers has an associated weight $w$. Let $w_s$ be a numeric weight associated with set $S$. Thus the the 'weight' of each alternative $a_i = s_i \in S$ can be given by $w_s * s_i^'$ (i.e.,$w_s * normalized \space value \space \forall s_i)$ (basically wish to perform simple additive weighting, for giving the process a name)

Question: Based on the situation, I'd want to to normalize within some interval - specifically [0,n] for some n (usually, 1, 10, 100). The question is then which method to use for normalization? Which is one 'correct' (if correct can by aptly defined in this context)? Are these all equivalent? (the first one doesn't have any max/min best/worst associated with it like the other two, implying that there IS probably a difference!)

PS: There is no associated tag for normalization and I've tried choosing the best ones describing the problem. Those with higher privileges my add/remove them to make it more appropriate

UPDATE: I'm looking to perform simple additive weighting - where scores/ratings that are on a different scale are normalized before proceeding. Given a set $C$ of criteria with weights $w_i$ and each alternative $a_j$ is rated on each criteria. The overall weight of each alternative is given by $\sum_{i=1}^n w_i * a_i$

   | C1 | C2 | W(C1) = 2 and W(C2) = 4
A1 | 20 |  7 |
A2 |  4 | 17 |
A3 | 12 |  3 |


The overall scores for the alternatives A1 - A3 would be calculated as per the above formula $\sum_{i=1}^n w_i * a_i$. The question is what normalizing (for the scores) approach is the best to use for this purpose? Does it matter?

(Note: The weights for each critieria $w_i$ are normalized using the first way as above)

-
Since you've said the components are all positive: 1) is normalization with respect to the Manhattan norm $\|\cdot\|_1$, while the first suggestion in 2) is normalizing with respect to the max-norm $\|\cdot\|_\infty$. Which of these is more appropriate is really dependent on the application. – J. M. Nov 13 '11 at 7:52
I don't think you've said enough about what you want to do with these numbers to define what it would mean for these normalization methods to be "equivalent". They're obviously not equivalent in the sense of immediately leading to the same result, so they could only be equivalent in some other sense, which I guess would have to involve leading to the same result in some later processing stage, which you'd have to describe. – joriki Nov 13 '11 at 7:52
@joriki - I've added the update – PhD Nov 13 '11 at 8:57

These methods are not equivalent. You still haven't specified exactly what you mean by "equivalent", but I presume it would include at least that they lead to the same order of the weighted scores, which they do not. All three of these methods (but not the part labeled "Another way" under 2) are linear, and they all have different scales. These scales effectively act as a further weighting. You can write all these transformations as

$$s'_{ij}=\alpha_j s_{ij}+\beta_j\;,$$

where $j$ labels the criterion and $i$ labels the value for that criterion (your indices and variables seem somewhat confused in this regard). Then the weighted score is

$$Q_i=\sum_j w_js'_{ij}=\sum_j w_j\left(\alpha_j s_{ij}+\beta_j\right)=\sum_jw_j\alpha_js_{ij}+\sum_jw_j\beta_j\;.$$

So the constants $\beta_j$ just shift all the scores by the same constant amount $\sum_jw_j\beta_j$, but the scalings $\alpha_j$ effectively change the weights from $w_j$ to $w_j\alpha_j$, and this will in general not just linearly transform the scores but may affect their order in different ways for different choices of $\alpha_j$.

In a sense there's no most appropriate choice, since you can choose the $w_j$ any way you like to make up for the choice of the $\alpha_j$, but I'll assume that the $w_j$ express something like how strongly you want the individual criteria to influence the result. Then the most appropriate choice of normalization will depend on the circumstances. For instance, if all the values for some criterion happen to be very similar in a particular case, you'd artificially enhance the small differences by using method 3. On the other hand, if it's in the nature of the criterion that its value varies only very slightly, and you want to pick up on that variation and weight it highly, then method 3 would be appropriate, whereas method 2 would treat the values to be almost the same. What you really want, I think, is something like method 3, but not using the minimal and maximal values of some given set of values that might vary more or less by coincidence, but using lower and upper ends of the spectrum of values that you'd expect for this criterion, so that $0$ would signify "worst possible value of this criterion" and $1$ "best possible value of this criterion", not just worst/best that happened to occur in this case.

-
That clarifies a lot of confusion! However, something is still nagging me though...what according to you, would be the most a appropriate situation for applying method #1? – PhD Nov 13 '11 at 20:44
@Nupul: The only situation where I can imagine method #1 making any sense is one where the criterion refers to some quantity that's shared among the items. For instance, if you're rating TV shows that are all on at the same time and you know how many people viewed each show, it might make sense to normalize those values as fractions of the total viewership of all the shows. Or it might be the number of people who gave this as their favourite item in a survey. Anything that adds up to a certain total, I guess. – joriki Nov 13 '11 at 20:57
aaah! That makes sense! Thanks a ton! – PhD Nov 13 '11 at 21:25
Why do you say that the method "Another way" is not linear? I didn't get that part...it doesn't seem to be 'non-linear'. Maybe I'm missing something...since 'another way' is to be done with method #2 for scales that mean 'lower the better' so I'm a bit confused with that – PhD Nov 13 '11 at 21:31
@Nupul: A transformation is linear if it's described by a linear polynomial, that is, $s'=\alpha s+\beta$ as in my answer. You can write each of your three transformations like this; for instance, for method $3$ (with rating $1$), $\alpha=-1/(\mathrm{worst}-\mathrm{best})$ and $\beta=\mathrm{worst}/(\mathrm{worst}-\mathrm{best})$. You can't write the method "Another way" like this, because $s$ is in the denominator. I'm not sure what you mean by this method being done with method #2; I'm basing this on the equation you wrote for this method, $s'_i=\min(S)/s_i$, with $s_i$ in the denominator. – joriki Nov 13 '11 at 22:13