# Stein's lemma condition

(Apologies if I break some conventions, this is my first time posting!)

I am working on proving Stein's characterization of the Normal distribution: for Z $\sim N(0,1)$ and some differentiable function $f$ with $E[|f'(Z)|] < \infty$, $$E[Zf(Z)] = E[f'(Z)]$$ Writing the LHS expression in integral form and integrating by parts, I eventually obtain: $$E[Zf(Z)] = \frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} + E[f'(Z)]$$ Now I need to show that the first expression on the right hand size is zero. Intuitively, this seems clear because of the exponential term, but I am having trouble explicitly applying the condition on $f'$ to prove this rigorously. Any ideas?

-
Do you have any control on the growth of $f$ at $\infty$? $\exp(-x^2)$ decays quite fast near $\infty$. – robjohn Jan 16 '13 at 21:44
As mentioned in the post, there is the condition that $E[|f'(z)|] < \infty$. It seems non-trivial to me to show that this restriction causes it to increase slower than the exponential term decreases... – gogurt Jan 16 '13 at 21:49

Given $E\left[ \lvert f'(z) \rvert \right] < \infty$, we want to show that $$\frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} = 0,$$ or alternatively that $$\lim_{z \to \infty} \bigg\{ f(-z) \exp\left(-z^2 / 2\right) - f(z) \exp\left(-z^2 / 2\right) \bigg\} = \lim_{z \to \infty} \bigg\{ \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\} = 0.$$

Since $f$ is differentiable everywhere, we have that $f(z) - f(-z) = \int_{-z}^z f'(x) dx.$ Then \begin{align} \bigg\lvert \frac{1}{\sqrt{2 \pi}} \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\rvert &= \bigg\lvert \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) \int_{-z}^z f'(x) dx \bigg\rvert \\ &= \bigg\lvert \int_{-z}^z \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) f'(x) dx \bigg\rvert \\ &= \bigg\lvert \int_{-z}^z f'(x) \, \phi(z) \, dx \bigg\rvert \\ &\le \int_{-z}^z \big\lvert f'(x) \, \phi(z) \big\rvert \, dx \\ &= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(z) \, dx \\ &= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(x)} \, dx \\ &\le \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(0)} \, dx \\ &= \frac{\phi(z)}{\phi(0)} \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \, dx. \end{align}

The first factor, $\phi(z) / \phi(0)$, has limit 0. The second factor has limit $E\Big[\big\lvert f'(x) \big\rvert\Big]$, which is finite. So their product has limit 0, as desired.

-
Hey Dougal! Thanks a lot for taking a look at this. This certainly does look good. You killed it. But one thing--what is the justification for your first step of rewriting the sum of the limits as the limit of the sum? It seems intuitively obvious but doesn't it require the assumption that the individual limits are finite? – gogurt Jan 17 '13 at 3:04
Hmm, I didn't think about that. But why is it the sum of the limits in the first place? Shouldn't it be the limit of the sum, if you think of the original integration by parts as $E\Big[ Z f(Z) \Big] = \lim_{z \to \infty} \int_{-z}^z x f(x) \phi(x)$? – Dougal Jan 17 '13 at 3:10
Hmm. That's interesting. I've never really interpreted it that way since I learned the derivation of integration by parts as an application of the product rule, in which case the first term in the integrand splits into the difference of two expressions, each evaluated at the respective point (i.e. each evaluated at its limit). In other words, $\int_a^b uv' = (uv)|_a^b - \int ...$ – gogurt Jan 17 '13 at 3:16
+1. I agree that it is actually the limit of the sum from the beginning because the expression comes from an improper integral of the form $\int_{-\infty}^\infty g(z)\,\mathrm dz$ which by definition is $\lim_{z\to\infty}\int_{-z}^z g(z)\,\mathrm dz$, right? – Stefan Hansen Jan 17 '13 at 6:54
I thought about it a bit more and I've been convinced. This is a great solution. Thanks Dougal! – gogurt Jan 17 '13 at 14:49

Let $Z\sim \mathcal{N}(0,1)$ and $f$ a differentiable function with $E[|f'(Z)|]<\infty$. Then \begin{align} E[Zf(Z)]&=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz-f(0)E[Z]\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[f(z)-f(0)\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[\int_0^zf'(u)\,\mathrm du\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz. \end{align} On the other hand \begin{align} E[f'(Z)]=&\int_{-\infty}^\infty f'(z)\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ =&\frac{1}{\sqrt{2\pi}}\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz\\ +&\frac{1}{\sqrt{2\pi}}\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz. \end{align} So let us treat these two integrals seperately and use Fubini's theorem (justified by the assumption): $$\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_0^\infty\int_z^\infty f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm du\,\mathrm dz\\ =\int_0^\infty \int_0^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du$$ and similarly \begin{align} &\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_{-\infty}^0\int_{u}^0f'(z)(-u)\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du\\ &=\int_{-\infty}^0\int_{0}^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du \end{align} and thus $$E[f'(Z)]=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}u\left[\int_{0}^u f'(z)\,\mathrm dz\right]\exp\left(-\frac{u^2}{2}\right)\,\mathrm du=E[Zf(Z)]$$

-
Thanks Stefan. That's an interesting way to do the problem that I hadn't thought of before. But might there be a way to use the condition on $f'$ to show that the limit expression in my original formulation is zero? I really only ask because in Casella and Berger they cryptically say "the condition on $f$ is enough to guarantee that the term is zero..." – gogurt Jan 16 '13 at 21:18
I'll have to think about this. Do we agree that the term being zero means that$$\lim_{z\to\infty,-\infty}f(z)\exp\left(-z^2/2\right)=0?$$ – Stefan Hansen Jan 16 '13 at 21:31
Yes, I agree. More or less it seems to be a matter of showing that the condition sufficiently limits the growth of $f(z)$ so that it doesn't dominate $\exp (-z^2/2)$ going to zero as $z \to \infty$. – gogurt Jan 16 '13 at 21:34

The condition $$\mathbb{E}[f'(x)] < \infty$$ tells you that $f$ is Lipschitz continuous $\mathbb{P}$-a.s. and thus a.s. locally bounded. You might then consider sequences of of functions $$f_n(x)= \left\{\begin{array}{lr} f(x): x \in [-n,n] \\ 0: \text{otherwise}\end{array}\right..$$ This is useful because you know that $f_n$ is a.s. bounded on $[-n,n]$ or $|f_n(x)| < M \cdot (2n)$ for $x \in [-n,n]$ and $M \ge 0$. If you then split $f_n$ into its positive and negative parts, you have that $f_n \le f_{n+1} \uparrow f$, allowing you to apply monotone convergence theorem so that you can pass to the limit. You might apply this to Dougal's $$\bigg\lvert \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) \int_{-z}^z f'(x) dx \bigg\rvert,$$ allowing you to avoid the subsequent computation.

-
Hey Soren, Thanks for chipping in! I'll have to think about it. There are some intermediate steps which I think need to be rigorously checked. Looks interesting though. – gogurt Jan 17 '13 at 14:40