If one is adding probabilities as log probabilities lp and lq so that $lsum =\max(lp,lq)+\log(1+\exp(-|lp-lq|))$ it would be nice to have a slightly less computationally expensive correction to the max term. I known that $\log(1+x) = x-x^2/2+x^3/3-x^4/4$ etc. However firstly that's not great for x=1 and also I would think there was more that could be done since $x = \exp(-|a|)$. Reasonable ranges of a are from 0 to 10. Any thoughts?

  • $\begingroup$ series expansion around $a=0$ for $log(1+exp(-a))$? check here $\endgroup$
    – Xminer
    May 28, 2019 at 7:38
  • $\begingroup$ I checked that series approximation but it requires very high order to not become very divergent for reasonable ranges of a. You can truncate to zero when a is reasonably large but I think that would be maybe for a>10 or so. Even 4th order is no good above a=2. $\endgroup$
    – Robotbugs
    May 28, 2019 at 9:51
  • 2
    $\begingroup$ How approximate can you get away with? You could write $\log(1+\exp(-|a|)) \sim \log(2)\exp(-|a|)$ for something that behaves similarly but is systematically smaller. You can add a constant $\log(1+\exp(-|a|)) \sim \log(2)\exp(-0.85|a|)$ and the approximation might be a little better. $\endgroup$ May 28, 2019 at 10:27
  • 1
    $\begingroup$ You could do your expansion around $a =5$ instead of $a = 0$. I.e., the midpoint of the range you want to compute over, not one end. $\endgroup$ May 28, 2019 at 20:56


You must log in to answer this question.

Browse other questions tagged .