15
$\begingroup$

I am trying to evaluate the following: The expectation of the hyperbolic tangent of an arbitrary normal random variable.

$\mathbb{E}[\mathrm{tanh}(\phi)]; \phi \sim N(\mu, \sigma^2)$

Equivalently:

$\int_{-\infty}^{\infty} \mathrm{tanh}(\phi)\frac{1}{\sqrt{2\pi\sigma^2}}\exp(-\frac{1}{2\sigma^2}(\phi-\mu)^2) d\phi$

I've resorted to Wolfram Alpha, and I can sometimes (!) get it to evaluate the integral for $\sigma^2 = 1$. It gives:

$\exp(\mu) - 1$ for negative $\mu$ and $1-\exp(-\mu)$ for positive mu.

I have no idea how it got this, but it seems plausible as I've done some simulations (i.e. lots of draws from the normal distribution and then the mean of the tanh of the draws) and the formula it gives for the $\sigma^2 = 1$ case seems pretty close.

I cannot get it to evalute anything with a different variance. Mathematica also seems unwilling to compute the integral or the Fourier transform (which I thought might be a way forward) but this is tricker than the ones I know how to deal with.

Thanks!

$\endgroup$
0
2
$\begingroup$

As a challenge for myself, I tried to do the maths. Unfortunately, I am not quite the math guru and I did not manage to find a solution (yet). Considering that my effort could help you on the way, I decided to share my non-result nevertheless:

$$\begin{align} \int_{-\infty}^{+\infty} \mathrm{tanh}(x)\cdot \frac{1}{\sqrt{2 \pi \sigma^2}} e^{-\frac{(x-\mu)^2}{2 \sigma^2}} \mathrm{d}x & = \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^{+\infty} \frac{e^x - e^{-x}}{e^x + e^{-x}} \cdot e^{-\frac{(x-\mu)^2}{2 \sigma^2}} \mathrm{d}x \\ & = \frac{1}{\sqrt{2 \pi \sigma^2}} \left( \int_{-\infty}^{+\infty} \frac{e^x}{e^x + e^{-x}} \cdot e^{-\frac{(x-\mu)^2}{2 \sigma^2}} \mathrm{d}x - \int_{-\infty}^{+\infty} \frac{e^{-x}}{e^x + e^{-x}} \cdot e^{-\frac{(x-\mu)^2}{2 \sigma^2}} \mathrm{d}x \right) \\ & = \frac{1}{\sqrt{2 \pi \sigma^2}} \left( \int_{-\infty}^{+\infty} \frac{1}{1 + e^{-2x}} \cdot e^{-\frac{(x-\mu)^2}{2 \sigma^2}} \mathrm{d}x - \int_{-\infty}^{+\infty} \frac{1}{e^{2x} + 1} \cdot e^{-\frac{(x-\mu)^2}{2 \sigma^2}} \mathrm{d}x \right) \\ \end{align}$$

Using the substitution $u = \frac{x-\mu}{\sqrt{2 \sigma^2}}$ and integration by parts where $\int \frac{1}{1 + e^{ax+b}} \mathrm{d}x = -\frac{1}{a} \ln\left(e^{-ax-b} + 1\right)$ , I managed to get to

$$\begin{align} \int_{-\infty}^{+\infty} \frac{1}{1 + e^{-2x}} e^{-\frac{(x-\mu)^2}{2 \sigma^2}} \mathrm{d}x & = \sqrt{2 \sigma^2}\int_{-\infty}^{+\infty} \frac{1}{1 + e^{-2(\sqrt{2 \sigma^2} u + \mu)}} e^{-u^2} \mathrm{d}u \\ & = \int_{-\infty}^{+\infty} \ln\left(e^{2(\sqrt{2 \sigma^2} u + \mu)} + 1\right) \; u e^{-u^2} \mathrm{d}u \end{align}$$

This is where I got to the point that my inspiration ran out and I considered I spent enough time on the problem. This expression is also still not simple enough for wolfram alpha to handle (unless you simplify to something like $\int_{-\infty}^{\infty} \ln\left(e^{2x} + 1\right)\;xe^{-x^2} \mathrm{d}x$).

BTW: If anyone would be able to finish this answer (or provide another approach), I would also be interested in the solution.

$\endgroup$
0
2
+100
$\begingroup$

$\def\f{\varphi} \def\s{\sigma} \def\m{\mu} \def\p{\pi}$Let $$f(\f) = \frac{1}{\sqrt{2\p}\s} e^{-(\f-\m)^2/(2\s^2)}.$$ We have \begin{align*} I &= \int_{-\infty}^\infty f(\f)\tanh\f\, d\f = \int_0^\infty f(\f)\tanh\f\, d\f + \int_{-\infty}^0 f(\f)\tanh\f\, d\f \\ &= \int_0^\infty (f(\f)-f(-\f))\tanh\f\, d\f. \end{align*} But \begin{align*} \tanh\f &= \frac{1-e^{-2\f}}{1+e^{-2\f}} = (1-e^{-2\f})\sum_{k=0}^\infty (-1)^k e^{-2k\f} \\ &= 1+2\sum_{k=1}^\infty (-1)^k e^{-2k\f}. \end{align*} It is a straightforward exercise to show that \begin{align*} J_k &= \int_0^\infty (f(\f)-f(-\f)e^{-2k\f} d\f \\ &= \frac12 e^{-2k(\m-k\s^2)} \left( 1+\mathrm{erf} \left(\frac{\m-2k\s^2}{\sqrt2 \s}\right) \right) - \frac12 e^{2k(\m+k\s^2)} \mathrm{erfc}\left(\frac{\m+2k\s^2}{\sqrt2\s}\right). \end{align*} One can show for large $k$ that $J_k$ is of order $1/k^2$. Let $$I_n = J_0 + 2\sum_{k=1}^n (-1)^k J_k.$$ Thus, $I = \lim_{n\to\infty}I_n.$ The zeroth approximation is $$I_0 = J_0 = \mathrm{erf}\left(\frac{\m}{\sqrt{2}\s}\right).$$ (This amounts to approximating $\tanh\f$ by $\mathrm{sign}(\f)$ in the defining integral.) Below we plot $I_0$, $I_1 = J_0 - 2J_1$, and $\hat I$, an independent numerical approximation to $I$, as functions of $\m$ for $\s=1$. On this scale $I_2$ would be indistinguishable from $\hat I$.

enter image description here

Figure 1. $I_0$, $I_1$, and $\hat I$ (in red, orange, and black, respectively) vs $\m$ for $\s=1$.

Addendum: Interchanging summation and integration

Let $g(\f) = f(\f)-f(-\f)$. Note that $$g(\f)\sum_{k=1}^\infty (-1)^k e^{-2k \f}$$ converges pointwise to $$-\frac{g(\f)}{1+e^{2\f}}$$ on $(0,\infty)$. Also, $$\left| g(\f)\sum_{k=1}^n (-1)^k e^{-2k \f} \right| \le \left|g(\f)\right|.$$ Since $|g(\f)|$ is integrable, interchanging summation and integration is justified by the dominated convergence theorem.

$\endgroup$
4
  • $\begingroup$ Oh, I had the vague idea to write tanh as a series but I didn't know if the convergence held. But since we are integrating only nonnegative, we have convergence everywhere? $\endgroup$
    – qwr
    May 15 '21 at 0:20
  • $\begingroup$ Why does the integral over the negative reals cancel out? $\endgroup$ May 15 '21 at 13:54
  • 1
    $\begingroup$ @TymaGaidash I believe a simple change of variable like $\phi \mapsto -\phi$ was used. Note $\tanh$ is an odd function, i.e. $\tanh(-\phi) = -\tanh(\phi)$. $\endgroup$
    – qwr
    May 16 '21 at 6:03
  • $\begingroup$ @qwr: I have added a justification of the interchange of limits above. $\endgroup$
    – user26872
    May 17 '21 at 15:32
0
$\begingroup$

Here is an idea for a "reparameterization trick": Let $\phi = \sigma z + \mu$ where $z \sim N(0,1)$.

Then $\operatorname{E}[\tanh \phi] = \operatorname{E}[\tanh(\sigma z + \mu)]$. There is the hyperbolic tangent sum rule

$$ \operatorname{E}[\tanh(\sigma z+\mu)] = \operatorname{E} \left [\frac{\tanh (\sigma z) + \tanh \mu}{1 + \tanh (\sigma z) \tanh \mu} \right] $$

where $\tanh \mu$ is constant. Maybe there is a nice way to evaluate this. Note $\operatorname{E}[\tanh (\sigma z)] = 0$ since $\sigma z$ and $\tanh$ are symmetric.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.