Stein's lemma condition (Apologies if I break some conventions, this is my first time posting!)
I am working on proving Stein's characterization of the Normal distribution: for Z $\sim N(0,1)$ and some differentiable function $f$ with $E[|f'(Z)|] < \infty$, $$E[Zf(Z)] = E[f'(Z)]$$
Writing the LHS expression in integral form and integrating by parts, I eventually obtain:
$$E[Zf(Z)] = \frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} + E[f'(Z)]$$
Now I need to show that the first expression on the right hand size is zero. Intuitively, this seems clear because of the exponential term, but I am having trouble explicitly applying the condition on $f'$ to prove this rigorously. Any ideas?
 A: I know I'm a few years late to the party, but I'm not sure about Dougal's and soren's solutions.
In Dougal's solution, the replacement of $\phi(x)$ by its maximum $\phi(0)$ in the denominator results in a smaller quantity rather than a larger one.
In soren's solution, I don't understand why $f'(Z)$ having finite expectation would imply that $f$ is Lipschitz. What if $f'(z) = z$ for instance? Then $f'(Z)$ has finite expectation but $f(z) = z^2/2$ isn't Lipschitz.
Stefan's solution seems fine, but I agree with gogurt that a more "direct" attack on the product term of the integration by parts might be informative.
Almost every source I could find takes the Casella and Berger approach, saying something like "it can be shown that the product term is zero." Or some of them give Stefan's proof. Finally, I found some course notes from a class taught by Sourav Chatterjee with a terse proof that the product term is zero. See Lemma 2.
Edit: The idea from Chatterjee's notes seems to be that you can show that $\mathbb{E}|f(Z)|$ is finite, which means that its integrand $f(z) \phi(z)$ must approach zero as $z \rightarrow \pm \infty$ if those limits exist (according to this).
He first shows that $\mathbb{E} |Z f(Z)|$ is finite:
\begin{align*}
\int_{-\infty}^\infty |zf(z)| \phi(z) dz &\leq \int_{-\infty}^\infty |z| \left[|f(0)| + |f(z) - f(0)|\right] \phi(z) dz\\
 &\leq \int_0^\infty z \left[\int_0^z |f'(t)| dt\right] \phi(z) dz + \int_{-\infty}^0 (-z) \left[\int_z^0 |f'(t)| dt\right] \phi(z) dz + |f(0)| \sqrt{2/\pi}\\
 &= \int_0^\infty |f'(t)| \underbrace{\int_t^\infty z \phi(z) dz}_{\phi(t)} dt + \int_{-\infty}^0 |f'(t)|\underbrace{ \int_{-\infty}^t (-z) \phi(z) dz}_{\phi(t)} dt + |f(0)| \sqrt{2/\pi}\\
 &= \int_{\infty}^\infty |f'(t)| \phi(t) dt + |f(0)| \sqrt{2/\pi}\\
 &= \mathbb{E} |f'(Z)| + |f(0)| \sqrt{2/\pi}
\end{align*}
Finally, take expectations of both sides of the pointwise inequality
\begin{align*}
|f(Z)| \leq \sup_{|t| \leq 1} |f(t)| + |Zf(Z)|
\end{align*}
The continuity of $f$ ensures that its supremum on $[-1, 1]$ is finite.
A: Let $Z\sim \mathcal{N}(0,1)$ and $f$ a differentiable function with $E[|f'(Z)|]<\infty$. Then
$$
\begin{align}
E[Zf(Z)]&=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz-f(0)E[Z]\\
&=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[f(z)-f(0)\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\
&=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[\int_0^zf'(u)\,\mathrm du\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz.
\end{align}
$$
On the other hand
$$
\begin{align}
E[f'(Z)]=&\int_{-\infty}^\infty f'(z)\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\
=&\frac{1}{\sqrt{2\pi}}\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz\\
+&\frac{1}{\sqrt{2\pi}}\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz.
\end{align}
$$
So let us treat these two integrals seperately and use Fubini's theorem (justified by the assumption):
$$
\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_0^\infty\int_z^\infty f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm du\,\mathrm dz\\
=\int_0^\infty \int_0^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du
$$
and similarly
$$
\begin{align}
&\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_{-\infty}^0\int_{u}^0f'(z)(-u)\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du\\
&=\int_{-\infty}^0\int_{0}^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du
\end{align}
$$
and thus
$$
E[f'(Z)]=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}u\left[\int_{0}^u f'(z)\,\mathrm dz\right]\exp\left(-\frac{u^2}{2}\right)\,\mathrm du=E[Zf(Z)]
$$
A: This might be too late in the game but here is an answer to Gogurt's original question, that shows that the assumptions on $f$ imply that
$f(z)\phi(z)$ vanishes at infinity.
Student45 establishes that $f$ and $zf(z)$ are integrable with respect to the normal distribution using Fubini's theorem. This implies that $f'(z)\phi(z)+ f(z)z\phi(z)=(\phi(z)f(z))'$ is Lebesgue integrable in $\mathbb{R}$. In particular,
$$ f(z)\phi(z) = f(0)\phi(0) + \int^z_0 (\phi\cdot f)'(t)dt $$
for all $z$. This is due to the Fundamental Theorem of Calculus that says that if $f$ is differentiable in an interval $[a,b]$ and if $f'$ is Lebesgue integrable in $[a,b]$, then
$$f(x)-f(a)=\int^x_af'(t)\,dt,\quad a\leq x\leq b.$$
Therefore, the limits $\lim_{z\rightarrow\pm\infty}f(z)\phi(z)=A_{\pm}$ exist. As $f(z)\phi(z)$ is Lebesgue integrable in $\mathbb{R}$, then one must have that $A_{\pm}=0$ as Student45 mentioned in his comment above.
A: Given $E\left[ \lvert f'(z) \rvert \right] < \infty$,
we want to show that
$$\frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} = 0,$$
or alternatively that
$$
\lim_{z \to \infty} \bigg\{ f(-z) \exp\left(-z^2 / 2\right) - f(z) \exp\left(-z^2 / 2\right) \bigg\}
= \lim_{z \to \infty} \bigg\{ \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\}
= 0.
$$
Since $f$ is differentiable everywhere, we have that
$f(z) - f(-z) = \int_{-z}^z f'(x) dx.$
Then
$$
\begin{align}
\bigg\lvert \frac{1}{\sqrt{2 \pi}} \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\rvert
&= \bigg\lvert \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) \int_{-z}^z f'(x) dx \bigg\rvert \\
&= \bigg\lvert \int_{-z}^z \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) f'(x) dx \bigg\rvert \\
&= \bigg\lvert \int_{-z}^z f'(x) \, \phi(z) \, dx \bigg\rvert \\
&\le \int_{-z}^z \big\lvert f'(x)  \, \phi(z) \big\rvert \, dx \\
&= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(z) \, dx \\
&= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(x)} \, dx \\
&\le \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(0)} \, dx \\
&= \frac{\phi(z)}{\phi(0)} \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \, dx.
\end{align}
$$
The first factor, $\phi(z) / \phi(0)$, has limit 0.
The second factor has limit $E\Big[\big\lvert f'(x) \big\rvert\Big]$, which is finite.
So their product has limit 0, as desired.
A: The condition 
$$\mathbb{E}[f'(x)] < \infty$$
tells you that $f$ is Lipschitz continuous  $\mathbb{P}$-a.s. and thus a.s. locally bounded. You might then consider sequences of of functions
$$f_n(x)= \left\{\begin{array}{lr} f(x): x \in [-n,n] \\ 0: \text{otherwise}\end{array}\right..$$
This is useful because you know that $f_n$ is a.s. bounded on $[-n,n]$ or $|f_n(x)| < M \cdot (2n)$ for $x \in [-n,n]$ and $M \ge 0$. If you then split $f_n$ into its positive and negative parts, you have that $f_n \le f_{n+1} \uparrow f$, allowing you to apply monotone convergence theorem so that you can pass to the limit. You might apply this to Dougal's
$$\bigg\lvert \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) \int_{-z}^z f'(x) dx \bigg\rvert,$$
allowing you to avoid the subsequent computation.
A: If people in 2016 were late to the party, I think the party may have long since ended, but interestingly enough, nobody gave the proof using DCT yet, which I find to be the most direct and which I present here.
For any real $t$, note that
$$\left|f'(t)\exp(-z^2/2)\cdot 1_{[0,z]}(t)\right| \leq |f'(t)|\exp(-t^2/2)$$
The dominating function is independent of $z$ and is integrable by assumption. Also, we have the pointwise limit $$\lim_{z\to\infty} f'(t) \exp(-z^2/2) \cdot 1_{[0,z]}(t) = 0.$$
Therefore, by the Dominated Convergence Theorem, $$\lim_{z\to\infty} f(z)\exp(-z^2/2) = \lim_{z\to\infty}(f(z)-f(0))\exp(-z^2/2) = \lim_{z\to\infty}\int_0^z f'(t)\exp(-z^2/2)\,dt = 0.$$

The other limit, as $z\to-\infty$, is similar, using $$\left|f'(t)\exp(-z^2/2)\cdot 1_{[z,0]}(t)\right| \leq |f'(t)|\exp(-t^2/2)$$ instead.
A: student45's answer is great. I just fill some holes. The following is a complete and easy version of proof of Stein's lemma. Anyone familiar with calculus and intuitive probability theory is able to understand the proof.
Stein's Lemma [Casella and Berger (2002, p.124)]

Let $ X\sim N(\mu, \sigma^2) $, and let $ g $ be a differentiable function satisfying $\mathbb{E}|g'(X)|<+\infty$. Then $ \mathbb{E}[(X-\mu)g(X)] = \sigma^2 \mathbb{E}[g'(X)] $.

The proof consists of five steps.
(1) We show $\mathbb{E}|(X-\mu) g(X)| < +\infty$.
Let $ f(x)=\frac{1}{\sqrt{2\pi} \sigma} e^{-\frac{(x-\mu)^2}{2\sigma^2}} $.
\begin{align*}
  \mathbb{E}|(X-\mu) g(X)| &=\int_{-\infty}^{+\infty} |x-\mu| |g(\mu) + g(x)-g(\mu)| f(x)dx \\ 
  &\leq \int_{-\infty}^{+\infty} |x-\mu| |g(\mu)| f(x)dx 
  + \int_{-\infty}^{+\infty} |x-\mu| |g(x)-g(\mu)| f(x)dx.
\end{align*}
Let $ z=x-\mu $. The first integral of the right-hand side becomes
\begin{align*}
\int_{-\infty}^{+\infty} |x-\mu| |g(\mu)| f(x)dx &=|g(\mu)| \int_{-\infty}^{+\infty} |z| \frac{1}{\sqrt{2\pi} \sigma} e^{-\frac{z^2}{2\sigma^2}} dz \\ 
&=2|g(\mu)| \int_{0}^{+\infty} z \frac{1}{\sqrt{2\pi} \sigma} e^{-\frac{z^2}{2\sigma^2}} dz \\ 
&=|g(\mu)| \sigma \sqrt{\frac{2}{\pi}} .
\end{align*}
By $ |g(x)-g(\mu)| = |\int_\mu^x g'(u)du| \leq \int_\mu^x |g'(u)| du $, the second integral of the right-hand side becomes
\begin{align*}
  &\int_{-\infty}^{+\infty} |x-\mu| |g(x)-g(\mu)| f(x)dx \\ 
  &\leq \int_{-\infty}^{+\infty} |x-\mu| \left(\int_\mu^x |g'(u)| du\right) f(x)dx \\ 
  &= \int_{\mu}^{+\infty} dx \int_\mu^x (x-\mu) |g'(u)| f(x) du -  
  \int_{-\infty}^\mu dx \int_\mu^x (x-\mu) |g'(u)| f(x) du \\ 
  &=\lim_{M\to +\infty} \int_{\mu}^{M} dx \int_\mu^x (x-\mu) |g'(u)| f(x) du -  
  \lim_{m\to -\infty} \int_{m}^\mu dx \int_\mu^x (x-\mu) |g'(u)| f(x) du \\ 
  &=\lim_{M\to +\infty} \int_{\mu}^{M} du \int_{u}^M (x-\mu) |g'(u)| f(x) dx -  
  \lim_{m\to -\infty} \int_{m}^\mu du \int_m^u (x-\mu) |g'(u)| f(x) dx. 
\end{align*}
In the above, we interchange the order of integration.
\begin{align*}
  \lim_{M\to +\infty} \int_{\mu}^{M} du \int_{u}^M (x-\mu) |g'(u)| f(x) dx 
  &=\lim_{M\to +\infty} \int_{\mu}^{M} |g'(u)| du \int_{u}^M (x-\mu)  f(x) dx \\ 
  &=\lim_{M\to +\infty} \int_{\mu}^{M} |g'(u)| \frac{-\sigma}{\sqrt{2\pi}} \left[e^{-\frac{(M-\mu)^2}{2\sigma^2}} - e^{-\frac{(u-\mu)^2}{2\sigma^2}} \right]du \\ 
  &\leq \lim_{M\to +\infty} \int_{\mu}^{M} |g'(u)| \frac{\sigma}{\sqrt{2\pi}}  e^{-\frac{(u-\mu)^2}{2\sigma^2}} du \\ 
  &=\sigma^2 \int_{\mu}^{+\infty} |g'(u)| f(u) du \\
  &\leq \sigma^2 \mathbb{E}|g'(X)| <+\infty. \\ 
  % 
  -\lim_{m\to -\infty} \int_{m}^\mu du \int_m^u (x-\mu) |g'(u)| f(x) dx 
  &=-\lim_{m\to -\infty} \int_{m}^\mu |g'(u)| du \int_m^u (x-\mu)  f(x) dx \\ 
  &=\lim_{m\to -\infty} \int_{m}^\mu |g'(u)| \frac{\sigma}{\sqrt{2\pi}} \left[e^{-\frac{(u-\mu)^2}{2\sigma^2}} - e^{-\frac{(m-\mu)^2}{2\sigma^2}} \right]du \\ 
  &\leq \lim_{m\to -\infty} \int_{m}^\mu |g'(u)| \frac{\sigma}{\sqrt{2\pi}}  e^{-\frac{(u-\mu)^2}{2\sigma^2}} du \\
  &\leq \sigma^2 \mathbb{E}|g'(X)| <+\infty.
\end{align*}
Therefore, $\int_{-\infty}^{+\infty} |x-\mu| |g(x)-g(\mu)| f(x)dx \leq 2\sigma^2 \mathbb{E}|g'(X)|$, which implies
\begin{align*}
  \mathbb{E}|(X-\mu) g(X)| 
  &\leq \int_{-\infty}^{+\infty} |x-\mu| |g(\mu)| f(x)dx 
  + \int_{-\infty}^{+\infty} |x-\mu| |g(x)-g(\mu)| f(x)dx \\ 
  &= |g(\mu)| \sigma \sqrt{\frac{2}{\pi}} + \int_{-\infty}^{+\infty} |x-\mu| |g(x)-g(\mu)| f(x)dx \\ 
  &\leq  |g(\mu)| \sigma \sqrt{\frac{2}{\pi}} + 2\sigma^2 \mathbb{E}|g'(X)| <+\infty. 
\end{align*}
Obviously, $\mathbb{E}|(X-\mu) g(X)| <+\infty$ implies $\mathbb{E}[(X-\mu) g(X)] $ is finite.
(2) We show $\displaystyle\lim_{x\to-\infty}  g(x) f(x) $ and $\displaystyle\lim_{x\to+\infty}  g(x) f(x) $ exist and are finite. For every finite $m, M\in\mathbb{R}$ with $m<M$, let $ I(m,M)=\int_{m}^{M} (x-\mu)g(x)f(x)dx $.
\begin{align*}
  I(m,M)&=\int_{m}^{M} (x-\mu)g(x)\frac{1}{\sqrt{2\pi} \sigma} e^{-\frac{(x-\mu)^2}{2\sigma^2}}dx\\
  &=\int_m^M \frac{-\sigma^2}{\sqrt{2\pi} \sigma} g(x) d e^{-\frac{(x-\mu)^2}{2\sigma^2}} \\ 
  &=\left[\frac{-\sigma^2}{\sqrt{2\pi} \sigma} g(x)  e^{-\frac{(x-\mu)^2}{2\sigma^2}}\right]_m^M 
  - \int_m^M \frac{-\sigma^2}{\sqrt{2\pi} \sigma} g'(x)  e^{-\frac{(x-\mu)^2}{2\sigma^2}} dx \\ 
  &=\left[-\sigma^2 g(x) f(x)\right]_m^M 
  +\sigma^2 \int_m^M  g'(x)  f(x) dx. \\ 
  \left[-\sigma^2 g(x) f(x)\right]_m^M &= I(m,M) - \sigma^2 \int_m^M  g'(x)  f(x) dx. 
\end{align*}
Let $ m\to -\infty $ and $ M \to +\infty $. The limit of the above equation becomes
\begin{align*}
  \lim_{m\to-\infty \atop M\to +\infty} \left[-\sigma^2 g(x) f(x)\right]_{m}^{M} 
  &=\lim_{m\to-\infty \atop M\to +\infty} I(m,M) - \sigma^2 \int_{-\infty}^{+\infty}  g'(x)  f(x) dx. 
\end{align*}
Since $\displaystyle\lim_{m\to-\infty \atop M\to +\infty} I(m,M)=\mathbb{E}[(X-\mu) g(X)]$, which is finite, and $\mathbb{E}|g'(X)|<+\infty$ implies $\mathbb{E}[g'(X)]$ is finite, we are sure that $\displaystyle\lim_{m\to-\infty \atop M\to +\infty} \left[-\sigma^2 g(x) f(x)\right]_{m}^{M} $ exists and is finite. That is, $\displaystyle\lim_{x\to-\infty}  g(x) f(x) $ and $\displaystyle\lim_{x\to+\infty}  g(x) f(x) $ exist and are finite.
(3) We show $\mathbb{E}|g(X)| <+\infty$. It is clear that $|g(x)| \leq \sup_{|x-\mu|\leq 1} |g(x)|$ for all $ x $ satisfying $ |x-\mu|\leq 1 $. For all $ x $ satisfying $ |x-\mu|>1 $, $|x-\mu| |g(x)| > |g(x)|$. Combining two cases gives
\begin{align*}
|g(X)| \leq \sup_{|X-\mu|\leq 1} |g(X)| + |(X-\mu)g(X)|.
\end{align*}
Taking expectations of both sides gives
\begin{align*}
  \mathbb{E}|g(X)| \leq \sup_{|x-\mu|\leq 1} |g(x)| + \mathbb{E}|(X-\mu)g(X)| <+\infty.
\end{align*}
(4) Finite $\mathbb{E}|g(X)|$ implies $\lim_{x\to-\infty}  g(x) f(x) =\lim_{x\to+\infty}  g(x) f(x) =0$. If they fail, $\mathbb{E}|g(X)|$ would be infinite. Actually, let $\lim_{x\to+\infty}  g(x) f(x) = a$. For every $\varepsilon>0$, there exists $ t\in\mathbb{R} $ such that $a-\varepsilon < g(x)f(x)<a+\varepsilon$ for all $ x>t $. If $a>0$, we can choose $\varepsilon$ satisfying $a-\varepsilon>0$. In this case,
\begin{align*}
  \mathbb{E}|g(X)| &= \int_{-\infty}^{+\infty} |g(x)| f(x)dx \\ 
  &=\int_{-\infty}^t |g(x)f(x)|dx + \int_t^{+\infty} |g(x)f(x)|dx \\ 
  &>\int_{-\infty}^t |g(x)f(x)|dx + \int_t^{+\infty} (a-\varepsilon)dx,
\end{align*}
which is infitite. If $ a<0 $, we choose $ \varepsilon>0 $ such that $ a+\varepsilon<0 $.
\begin{align*}
  \mathbb{E}|g(X)|  
  &=\int_{-\infty}^t |g(x)f(x)|dx + \int_t^{+\infty} |g(x)f(x)|dx \\ 
  &>\int_{-\infty}^t |g(x)f(x)|dx - \int_t^{+\infty} (a+\varepsilon)dx,
\end{align*}
which is also infinite. Therefore, $\lim_{x\to+\infty}  g(x) f(x)=0$. Similarly, $\lim_{x\to-\infty}  g(x) f(x)=0$ must hold.
Generally, finite $\mathbb{E}|g(X)|$ does not necessarily implies $\lim_{x\to \infty} |g(x)f(x)|=0$ since the limit may not exist.
(5) Stein's lemma follows. In step 2, we have shown that
\begin{align*}
  I(m,M)
  &=\left[-\sigma^2 g(x) f(x)\right]_m^M   +\sigma^2 \int_m^M  g'(x)  f(x) dx. 
\end{align*}
Let $ m\to -\infty $ and $ M \to +\infty $. We have
\begin{align*}
\mathbb{E}[(X-\mu)g(X)] = 0+ \sigma^2 \int_{-\infty}^{+\infty}  g'(x)  f(x) dx
=\sigma^2 \mathbb{E}[g'(X)].
\end{align*}
Q.E.D.
