Consider a positive function $f(x)$ and suppose that we would like to approximate its value around some point $x_0$. One way to do so is to use two-term Taylor series expansion as follows.

$$ f(x) \doteq f(x_0) + f'(x_0)(x-x_0) + \frac{1}{2}f''(x_0)(x-x_0)^2. $$

However, I read the following result where it is assumed that $f(x) = e^{h(x)}$ for some differentiable $h(x)$ and the new expansion is as follows.

$$ f(x) \doteq \exp \left\{ h(x_0) + h'(x_0)(x-x_0) + \frac{1}{2}h''(x_0)(x-x_0)^2 \right\}. $$

I do not think the second formula is correct. Let us check.

$$ f'(x) = e^{h(x)}h'(x) \Rightarrow f'(x_0) = e^{h(x_0)}h'(x_0); $$


$$ f''(x) = e^{h(x)}h'(x)h'(x) + e^{h(x)}h''(x) \Rightarrow f''(x_0) = e^{h(x_0)}h'(x_0)h'(x_0) + e^{h(x_0)}h''(x_0). $$

Plugging $f'(x_0)$ and $f''(x_0)$ into the first formula will not give the second one.

Update: I guess what they did was to expand $h(x)$ at $x_0$ first, that is,

$$ h(x) \doteq h(x_0) + h'(x_0)(x-x_0) + \frac{1}{2}h''(x_0)(x-x_0)^2. $$

Then they replace $h(x)$ by the above approximation. Last by the continuity of the exponential function, the approximation would hold.

Of course this form is easier. My question is how often this kind of approximation is used. Apart from simplicity, why would people use this, please? Thank you!


It does if you further expand the exponential : $$ \exp(Z) = 1 + Z + Z^2/2 + o(Z^2) .$$ Write $Z = h'(x_0)(x-x_0) + h''(x_0)(x-x_0)^2/2$, so that $Z\to0$ and $$ \exp\{h(x_0) + h'(x_0)(x-x_0) + h''(x_0)(x-x_0)^2/2 \} = e^{h(x_0)}\exp\{Z\} = e^{h(x_0)}(1+Z+Z^2/2 + o(Z)) .$$ Now replace $Z$ by its definition, expand everything and ignore terms that are $o(x-x_0)^2$ : you should obtain the first expression, that is, $$e^{h(x_0)}\big(1 + h'(x_0)(x-x_0) + (h''(x_0) + h'(x_0)^2)(x-x_0)^2/2 + o((x-x_0)^2)\big) $$ which agrees with the expressions of $f'$ and $f''$ in terms of $h$.

To answer your second question : there are contexts in which it is more useful to work with $h(x)$ than $f(x)$, one of the most important being Laplace's method for estimating integrals (or the saddle-point method). The reason it is sometimes more convenient to work with $h(x)$ than with $f(x)$, is because it often happens that the integrals one wants to evaluate asymptotically look a lot like $$ \int_0^\infty g(x) f(x)^N {\rm d}x ,$$ and you want to know what happens as $N\to\infty$. For instance, $n! = \int_0^\infty e^{-t}t^n{\rm d}t$ is one of them. Now the taylor series for $f(x)^N$ is a mess, but if you write $f(x)^N = e^{Nh(x)}$, then you can inject the Taylor series for $h(x)$ (at least around some point $x_0$) and just multiply by $N$ : things are a lot easier to work with.

  • $\begingroup$ Thanks. Please see my update. Do you mean the two forms are equivalent, please? That is, the approximation errors from the two are the same? $\endgroup$ – LaTeXFan Feb 3 '15 at 0:27
  • $\begingroup$ Yes : they are the same up to an error of $o((x-x_0)^2)$. $\endgroup$ – Sary Feb 3 '15 at 1:41
  • $\begingroup$ I've made an edit to try to answer your second question. $\endgroup$ – Sary Feb 3 '15 at 1:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.