How to compute $\mathbb{E}_X \log(1+\exp(X))$ where $X \sim \mathcal{N}(\mu,\sigma^2)$?

Problem

Obtain: $$\mathbb{E}_X \log(1+\exp(X))$$ for $$X \sim \mathcal{N}(\mu,\sigma^2)$$.

Definitions

• Logistic function: $$\displaystyle \mathcal{L}(x) := \frac{1}{1+\exp(-x)}$$
• Logit function: $$\displaystyle \mathcal{L}^{-1}(y) := \log \left( \frac{y}{1-y} \right)$$

Try

First let $$\displaystyle Y\equiv -\log \left( \frac{1}{1+\exp(X)} \right) = -\log\left(\mathcal{L} (-X)\right)$$. By change-of-variable,

\begin{aligned} f_Y(y) &= f_X(x) \left\vert \frac{\partial x}{\partial y} \right\vert\\[8pt] &= \frac{1}{\sigma\sqrt{2\pi}} \exp\left( - \frac{(x-\mu)^2}{2\sigma^2} \right) \vert 1 + \exp(-x) \vert \\[8pt] &= \frac{1}{\sigma\sqrt{2\pi}} \exp\left( - \frac{(\mathcal{L}^{-1}\left[ \exp(-y) \right]+\mu)^2}{2\sigma^2} \right) \frac{I(y>0)}{1-\exp(-y)} \ \ \because x = -\mathcal{L}^{-1}\left[ \exp(-y) \right] \\[8pt] \end{aligned}

Next, let us take the expectation

\begin{aligned} \mathbf{E}_Y[Y] &= \int_0^\infty y f_Y(y) dy \\[8pt] &= \frac{1}{\sigma\sqrt{2\pi}} \int_0^\infty \frac{y}{1-\exp(-y)} \exp\left( - \frac{(\mathcal{L}^{-1}\left[ \exp(-y) \right]+\mu)^2}{2\sigma^2} \right) dy \end{aligned}

Let $$w:=-\mathcal{L}^{-1}\left[ \exp(-y) \right]$$, then

$$dw = \exp(-y) \left( \frac{1}{\exp(-y)} + \frac{1}{1-\exp(-y)} \right) dy = \frac{dy}{1-\exp(-y)}$$

which circularly begs the original question as

\begin{aligned} \mathbf{E}_Y[Y] &= \frac{1}{\sigma\sqrt{2\pi}} \int_{-\infty}^\infty \log(1+\exp(w)) \exp \left( -\frac{(w-\mu)^2}{2\sigma^2} \right) dw \\[8pt] &= \mathbb{E}_X \log(1+\exp(X)) \end{aligned}

• I'm not sure if you can get the exact solution... If you just want an approximate value , there is an approximation for $1+Y$, where $Y$ follows a lognormal distribution. May 23 '20 at 11:47
• @Babado Can you be more specific? Currently I could not relate my question to lognormal distribution... Approximate value is satisfactory for me. May 23 '20 at 11:50
• if $X \sim N(\mu,\sigma^2)$ then $Y=e^X \sim LogNormal(\mu,\sigma^2)$ and you can approximate $Y+1$ which follows a three parameter lognormal distribution. May 23 '20 at 12:45
• A few days ago I had a problem similar to yours and I searched for an approximation of $Log(1+e^X)$ and I found something... Now I was trying to find it again but I can't find it :( . But I guess it had something to do with Taylor expansion and Monte-Carlo simulation... I'm sorry I can't help you more. However, if you have $\mu=0$, Andrew's deduction is very nice. May 23 '20 at 13:08
• I don't have access to a keyboard to LaTeX this up, but here's a series that quickly converges to the desired value, a few terms gives a decent approximation. May 23 '20 at 13:59

$$\mathbb{E}[\log(1+Y)]\approx \log(1+\mathbb{E}[Y])-\frac{\operatorname{Var}(Y)}{2(1+\mathbb{E}[Y])^2}$$ In our case, since $$Y = e^X \sim \operatorname{LogNormal(}\mu,\sigma^2)$$, we get that $$\mathbb{E}[\log(1+e^X)] \approx \log(1+e^{\mu+\frac12 \sigma^2})-\frac{(e^{\sigma^2}-1)e^{2 \mu + \sigma^2}}{2(1+e^{\mu+\frac12 \sigma^2})^2}$$