# Stein's lemma condition

(Apologies if I break some conventions, this is my first time posting!)

I am working on proving Stein's characterization of the Normal distribution: for Z $\sim N(0,1)$ and some differentiable function $f$ with $E[|f'(Z)|] < \infty$, $$E[Zf(Z)] = E[f'(Z)]$$ Writing the LHS expression in integral form and integrating by parts, I eventually obtain: $$E[Zf(Z)] = \frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} + E[f'(Z)]$$ Now I need to show that the first expression on the right hand size is zero. Intuitively, this seems clear because of the exponential term, but I am having trouble explicitly applying the condition on $f'$ to prove this rigorously. Any ideas?

• Do you have any control on the growth of $f$ at $\infty$? $\exp(-x^2)$ decays quite fast near $\infty$. – robjohn Jan 16 '13 at 21:44
• As mentioned in the post, there is the condition that $E[|f'(z)|] < \infty$. It seems non-trivial to me to show that this restriction causes it to increase slower than the exponential term decreases... – gogurt Jan 16 '13 at 21:49

I know I'm a few years late to the party, but I'm not sure about Dougal's and soren's solutions.

In Dougal's solution, the replacement of $\phi(x)$ by its maximum $\phi(0)$ in the denominator results in a smaller quantity rather than a larger one.

In soren's solution, I don't understand why $f'(Z)$ having finite expectation would imply that $f$ is Lipschitz. What if $f'(z) = z$ for instance? Then $f'(Z)$ has finite expectation but $f(z) = z^2/2$ isn't Lipschitz.

Stefan's solution seems fine, but I agree with gogurt that a more "direct" attack on the product term of the integration by parts might be informative.

Almost every source I could find takes the Casella and Berger approach, saying something like "it can be shown that the product term is zero." Or some of them give Stefan's proof. Finally, I found some course notes from a class taught by Sourav Chatterjee with a terse proof that the product term is zero. See Lemma 2.

Edit: The idea from Chatterjee's notes seems to be that you can show that $\mathbb{E}|f(Z)|$ is finite, which means that its integrand $f(z) \phi(z)$ must approach zero as $z \rightarrow \pm \infty$ if those limits exist (according to this).

He first shows that $\mathbb{E} |Z f(Z)|$ is finite: \begin{align*} \int_{-\infty}^\infty |zf(z)| \phi(z) dz &\leq \int_{-\infty}^\infty |z| \left[|f(0)| + |f(z) - f(0)|\right] \phi(z) dz\\ &\leq \int_0^\infty z \left[\int_0^z |f'(t)| dt\right] \phi(z) dz + \int_{-\infty}^0 (-z) \left[\int_z^0 |f'(t)| dt\right] \phi(z) dz + |f(0)| \sqrt{2/\pi}\\ &= \int_0^\infty |f'(t)| \underbrace{\int_t^\infty z \phi(z) dz}_{\phi(t)} dt + \int_{-\infty}^0 |f'(t)|\underbrace{ \int_{-\infty}^t (-z) \phi(z) dz}_{\phi(t)} dt + |f(0)| \sqrt{2/\pi}\\ &= \int_{\infty}^\infty |f'(t)| \phi(t) dt + |f(0)| \sqrt{2/\pi}\\ &= \mathbb{E} |f'(Z)| + |f(0)| \sqrt{2/\pi} \end{align*} Finally, take expectations of both sides of the pointwise inequality \begin{align*} |f(Z)| \leq \sup_{|t| \leq 1} |f(t)| + |Zf(Z)| \end{align*} The continuity of $f$ ensures that its supremum on $[-1, 1]$ is finite.

• Ugh, you're right that my answer from a few years ago was wrong. I think it's maybe salvageable (comment on the answer). But, it would be helpful if you brought the ideas of the proof from those course notes into this answer, since as it stands this isn't really an answer of its own. – Dougal Oct 31 '16 at 23:15
• This is the most direct answer thus far. Thanks, student45! And thanks for catching that error in the original answer. I glossed over it too quickly. – gogurt Jan 25 '17 at 14:01

Let $Z\sim \mathcal{N}(0,1)$ and $f$ a differentiable function with $E[|f'(Z)|]<\infty$. Then \begin{align} E[Zf(Z)]&=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz-f(0)E[Z]\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[f(z)-f(0)\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[\int_0^zf'(u)\,\mathrm du\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz. \end{align} On the other hand \begin{align} E[f'(Z)]=&\int_{-\infty}^\infty f'(z)\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ =&\frac{1}{\sqrt{2\pi}}\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz\\ +&\frac{1}{\sqrt{2\pi}}\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz. \end{align} So let us treat these two integrals seperately and use Fubini's theorem (justified by the assumption): $$\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_0^\infty\int_z^\infty f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm du\,\mathrm dz\\ =\int_0^\infty \int_0^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du$$ and similarly \begin{align} &\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_{-\infty}^0\int_{u}^0f'(z)(-u)\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du\\ &=\int_{-\infty}^0\int_{0}^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du \end{align} and thus $$E[f'(Z)]=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}u\left[\int_{0}^u f'(z)\,\mathrm dz\right]\exp\left(-\frac{u^2}{2}\right)\,\mathrm du=E[Zf(Z)]$$

• Thanks Stefan. That's an interesting way to do the problem that I hadn't thought of before. But might there be a way to use the condition on $f'$ to show that the limit expression in my original formulation is zero? I really only ask because in Casella and Berger they cryptically say "the condition on $f$ is enough to guarantee that the term is zero..." – gogurt Jan 16 '13 at 21:18
• I'll have to think about this. Do we agree that the term being zero means that$$\lim_{z\to\infty,-\infty}f(z)\exp\left(-z^2/2\right)=0?$$ – Stefan Hansen Jan 16 '13 at 21:31
• Yes, I agree. More or less it seems to be a matter of showing that the condition sufficiently limits the growth of $f(z)$ so that it doesn't dominate $\exp (-z^2/2)$ going to zero as $z \to \infty$. – gogurt Jan 16 '13 at 21:34

Given $E\left[ \lvert f'(z) \rvert \right] < \infty$, we want to show that $$\frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} = 0,$$ or alternatively that $$\lim_{z \to \infty} \bigg\{ f(-z) \exp\left(-z^2 / 2\right) - f(z) \exp\left(-z^2 / 2\right) \bigg\} = \lim_{z \to \infty} \bigg\{ \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\} = 0.$$

Since $f$ is differentiable everywhere, we have that $f(z) - f(-z) = \int_{-z}^z f'(x) dx.$ Then \begin{align} \bigg\lvert \frac{1}{\sqrt{2 \pi}} \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\rvert &= \bigg\lvert \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) \int_{-z}^z f'(x) dx \bigg\rvert \\ &= \bigg\lvert \int_{-z}^z \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) f'(x) dx \bigg\rvert \\ &= \bigg\lvert \int_{-z}^z f'(x) \, \phi(z) \, dx \bigg\rvert \\ &\le \int_{-z}^z \big\lvert f'(x) \, \phi(z) \big\rvert \, dx \\ &= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(z) \, dx \\ &= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(x)} \, dx \\ &\le \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(0)} \, dx \\ &= \frac{\phi(z)}{\phi(0)} \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \, dx. \end{align}

The first factor, $\phi(z) / \phi(0)$, has limit 0. The second factor has limit $E\Big[\big\lvert f'(x) \big\rvert\Big]$, which is finite. So their product has limit 0, as desired.

• Hey Dougal! Thanks a lot for taking a look at this. This certainly does look good. You killed it. But one thing--what is the justification for your first step of rewriting the sum of the limits as the limit of the sum? It seems intuitively obvious but doesn't it require the assumption that the individual limits are finite? – gogurt Jan 17 '13 at 3:04
• Hmm, I didn't think about that. But why is it the sum of the limits in the first place? Shouldn't it be the limit of the sum, if you think of the original integration by parts as $E\Big[ Z f(Z) \Big] = \lim_{z \to \infty} \int_{-z}^z x f(x) \phi(x)$? – Dougal Jan 17 '13 at 3:10
• Hmm. That's interesting. I've never really interpreted it that way since I learned the derivation of integration by parts as an application of the product rule, in which case the first term in the integrand splits into the difference of two expressions, each evaluated at the respective point (i.e. each evaluated at its limit). In other words, $\int_a^b uv' = (uv)|_a^b - \int ...$ – gogurt Jan 17 '13 at 3:16
• +1. I agree that it is actually the limit of the sum from the beginning because the expression comes from an improper integral of the form $\int_{-\infty}^\infty g(z)\,\mathrm dz$ which by definition is $\lim_{z\to\infty}\int_{-z}^z g(z)\,\mathrm dz$, right? – Stefan Hansen Jan 17 '13 at 6:54
• Oops! @student45 is right that my answer is incorrect. I think it should probably be salvageable by breaking the integral up into $[-z, -y], [-y, y], [y, z]$, with the central term going to zero because $\varphi(z) / \varphi(y) \to 0$ and the outer terms going to zero because $y$ to $z$ contains little probability mass. But I don't have time right now to make that rigorous.... – Dougal Oct 31 '16 at 23:13

This might be too late in the game but here is an answer to Gogurt's original question, that is show that the assumptions on $f$ imply that $f(z)\phi(z)$ vanishes at infinity.

Student45 establishes that $f$ and $zf(z)$ are integrable with respect to the normal distribution using Fubini's theorem. This implies that $f'(z)\phi(z)+ f(z)z\phi(z)=(\phi(z)f(z))'$ is Lebesgue integrable in $\mathbb{R}$. In particular, $$f(z)\phi(z) = f(0)\phi(0) + \int^z_0 (\phi\cdot f)'(t)dt$$ for all $z$. This is due to the Fundamental Theorem of Calculus that says that if $f$ is differentiable in an interval $[a,b]$ and if $f'$ is Lebesgue integrable in $[a,b]$, then $$f(x)-f(a)=\int^x_af'(t)\,dt,\quad a\leq x\leq b.$$ Therefore, the limits $\lim_{z\rightarrow\pm\infty}f(z)\phi(z)=A_{\pm}$ exist. As $f(z)\phi(z)$ is Lebesgue integrable in $\mathbb{R}$, then one must have that $A_{\pm}=0$ as Student45 mentioned in his comment above.

The condition $$\mathbb{E}[f'(x)] < \infty$$ tells you that $f$ is Lipschitz continuous $\mathbb{P}$-a.s. and thus a.s. locally bounded. You might then consider sequences of of functions $$f_n(x)= \left\{\begin{array}{lr} f(x): x \in [-n,n] \\ 0: \text{otherwise}\end{array}\right..$$ This is useful because you know that $f_n$ is a.s. bounded on $[-n,n]$ or $|f_n(x)| < M \cdot (2n)$ for $x \in [-n,n]$ and $M \ge 0$. If you then split $f_n$ into its positive and negative parts, you have that $f_n \le f_{n+1} \uparrow f$, allowing you to apply monotone convergence theorem so that you can pass to the limit. You might apply this to Dougal's $$\bigg\lvert \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) \int_{-z}^z f'(x) dx \bigg\rvert,$$ allowing you to avoid the subsequent computation.