2
$\begingroup$

Problem

Obtain: $\mathbb{E}_X \log(1+\exp(X))$ for $X \sim \mathcal{N}(\mu,\sigma^2)$.


Definitions

  • Logistic function: $\displaystyle \mathcal{L}(x) := \frac{1}{1+\exp(-x)}$
  • Logit function: $\displaystyle \mathcal{L}^{-1}(y) := \log \left( \frac{y}{1-y} \right)$

Try

First let $\displaystyle Y\equiv -\log \left( \frac{1}{1+\exp(X)} \right) = -\log\left(\mathcal{L} (-X)\right)$. By change-of-variable,

$$ \begin{aligned} f_Y(y) &= f_X(x) \left\vert \frac{\partial x}{\partial y} \right\vert\\[8pt] &= \frac{1}{\sigma\sqrt{2\pi}} \exp\left( - \frac{(x-\mu)^2}{2\sigma^2} \right) \vert 1 + \exp(-x) \vert \\[8pt] &= \frac{1}{\sigma\sqrt{2\pi}} \exp\left( - \frac{(\mathcal{L}^{-1}\left[ \exp(-y) \right]+\mu)^2}{2\sigma^2} \right) \frac{I(y>0)}{1-\exp(-y)} \ \ \because x = -\mathcal{L}^{-1}\left[ \exp(-y) \right] \\[8pt] \end{aligned} $$

Next, let us take the expectation

$$ \begin{aligned} \mathbf{E}_Y[Y] &= \int_0^\infty y f_Y(y) dy \\[8pt] &= \frac{1}{\sigma\sqrt{2\pi}} \int_0^\infty \frac{y}{1-\exp(-y)} \exp\left( - \frac{(\mathcal{L}^{-1}\left[ \exp(-y) \right]+\mu)^2}{2\sigma^2} \right) dy \end{aligned} $$

Let $w:=-\mathcal{L}^{-1}\left[ \exp(-y) \right]$, then

$$ dw = \exp(-y) \left( \frac{1}{\exp(-y)} + \frac{1}{1-\exp(-y)} \right) dy = \frac{dy}{1-\exp(-y)} $$

which circularly begs the original question as

$$ \begin{aligned} \mathbf{E}_Y[Y] &= \frac{1}{\sigma\sqrt{2\pi}} \int_{-\infty}^\infty \log(1+\exp(w)) \exp \left( -\frac{(w-\mu)^2}{2\sigma^2} \right) dw \\[8pt] &= \mathbb{E}_X \log(1+\exp(X)) \end{aligned} $$

$\endgroup$
7
  • 1
    $\begingroup$ I'm not sure if you can get the exact solution... If you just want an approximate value , there is an approximation for $1+Y$, where $Y$ follows a lognormal distribution. $\endgroup$
    – Babado
    May 23 '20 at 11:47
  • $\begingroup$ @Babado Can you be more specific? Currently I could not relate my question to lognormal distribution... Approximate value is satisfactory for me. $\endgroup$
    – Moreblue
    May 23 '20 at 11:50
  • 1
    $\begingroup$ if $X \sim N(\mu,\sigma^2)$ then $Y=e^X \sim LogNormal(\mu,\sigma^2)$ and you can approximate $Y+1$ which follows a three parameter lognormal distribution. $\endgroup$
    – Babado
    May 23 '20 at 12:45
  • 1
    $\begingroup$ A few days ago I had a problem similar to yours and I searched for an approximation of $Log(1+e^X)$ and I found something... Now I was trying to find it again but I can't find it :( . But I guess it had something to do with Taylor expansion and Monte-Carlo simulation... I'm sorry I can't help you more. However, if you have $\mu=0$, Andrew's deduction is very nice. $\endgroup$
    – Babado
    May 23 '20 at 13:08
  • 1
    $\begingroup$ I don't have access to a keyboard to LaTeX this up, but here's a series that quickly converges to the desired value, a few terms gives a decent approximation. $\endgroup$ May 23 '20 at 13:59
2
$\begingroup$

This question: Expected Value of the Logarithm might answer your problem.

Here it states that

$$\mathbb{E}[\log(1+Y)]\approx \log(1+\mathbb{E}[Y])-\frac{\operatorname{Var}(Y)}{2(1+\mathbb{E}[Y])^2}$$ In our case, since $Y = e^X \sim \operatorname{LogNormal(}\mu,\sigma^2)$, we get that $$\mathbb{E}[\log(1+e^X)] \approx \log(1+e^{\mu+\frac12 \sigma^2})-\frac{(e^{\sigma^2}-1)e^{2 \mu + \sigma^2}}{2(1+e^{\mu+\frac12 \sigma^2})^2}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.