How to compute ratios when the numbers are extremely small and numerical issue might arise? Consider that a set of numbers that can be written in the form $b^{x}$ (for some base):
$$A = \{ a_1, ..., a_n\}$$
for example maybe $\{ 2^{-2000}, 2^{-2002}, 2^{-2500}\}$. Also assume we can compute the numbers indirectly because we know its exponents (i.e. we are in log-space). I want to compute:
$$ \alpha_i = \frac{ a_i}{\sum^n_{i=1} a_i}$$
without running into numerical issues (the numerical issue arise because the original $a_i$'s are so small that the numerator and denominator are so close to zero that any reasonable computing object [with bounded precision $\epsilon$] has an error when trying to compute the ratio above. i.e. it approximately tries to compute $\frac{0}{0}$ and hence, an error). 
One way that I was told to do this is by "re-scaling" the numbers. In the sense, we can rescale A such that the largest has magnitude 1 and then perform the computation for $\alpha_i$. The mathematical correctness of the computation of alpha is obvious (with rescaling), as the max term divides top and bottom (cancels out and the ratio is computed correctly).
Let the one with the largest magnitude be denoted by $a^* = argmax_{a \in A}\{a\} $. Mathematically this would involve computing a new scaled set $A'$ as follow:
$$ A' = \{ \frac{a_1}{ a^*} , ..., \frac{a_n}{ a^*} \} = \{ a'_1 , ..., a'_n \}$$
my question is, why are we choosing to divide by the largest $a^*$ and not the smallest? Another way to view this could be, if they can all be written as an exponent, we are choosing the one with the most positive exponent. What is the issue with choosing the most negative exponent. (obviously, we wouldn't compute the division directly and instead compute the difference of the exponents, as to avoid the potential overflow issue, i.e. working in logspace and hence the exponents directly, so that we don't have the overflow issue when doing $\frac{a_i}{ a^*}$).
My intuition tells me that we actually should be dividing by the smallest, since that is the one that might be too small for our computations to handle (and hence the one we want to normalize). Consider it like this, say we write all the numbers in $A$ is exponential form, i.e. $a_i = 2^{e_i}$. Assume $e_i \leq 0$. Then to resolve the issue we can do:
$$ 2^{e_i - y}$$
where we want to make $e_i - y$ less negative (so that the exponentiation does not drive us to close to zero). To do that we want $-y$ to be very positive. So in this goal we choose $y$ to be the most negative exponent we can have. Hence, we choose the smallest exponent that corresponds to the smallest $a_i$. Thats what makes sense to me. But does choosing the max also work? If it does, why does it?
In the $A = \{ 2^{-2000}, 2^{-2002}, 2^{-2500}\}$ and $A' = \{ 1, 2^{-2}, 2^{-500}\}$ and hence, if the precision of our computer is $2^{-1023}$ to $2^{1023}$, this trick would seem to work. However, I wasn't sure if this trick would always work or what conditions it would work or why choosing the largest was a good idea and not the smallest. 
 A: When you are dealing with positive quantities, all you want is to keep all the numbers in range.  Your original examples are outside the assumed range of your computer of $2^{-1023}$ to $2^{1023}$ so you can't add them.  Any rescaling that brings them all into range will work just fine.  If you rescale by the smallest you get $A''=\{2^{500},2^{498},1\}$, which are all in range.  If your original data were wider ranging, it would be best to rescale by the middle of the range:  the geometric mean of the max and min.  So if $B=\{2^{-2000}, 2^{-2500}, 2^{-4000}\}$, rescaling by the max gives $B'=\{1,2^{-500},2^{-2000}\}$ which doesn't get them all into range.  Rescaling by the min doesn't either, but by the middle gives $B''=\{2^{1000},2^{500},2^{-1000}\}$ which works fine.  
When you are dealing with some positive and some negative quantities, you have o worry about subtractive cancellation, which is another issue.
A: You mention the potential issue of overflowing and that is exactly the problem. In the general case you won't get the nice $b^x$ for some basis $b$ for every numbers, and so you can't simply do calculations on the exponents.
Usually, rescaling in terms of the largest number works. If you have repeated arithmetical operations and run into problems with this, you can perform multiple rescalings. This is very common in the normalization step of the Baum-Welch algorithm, for example.
