derivative of expected value with respect to parameter in both pdf and expectation Say $X \sim N(\mu, \sigma^2)$ with pdf $f(x, \mu)$. We are interested in expectation of $g(x)$. Then
$$E[g(x, \mu)] = \int_{-\infty}^{\infty} g(x, \mu) f(x, \mu) dx$$
Now I want partial derivative of this. Why does the following hold?
$$\frac{d}{d\mu}E[g(x, \mu)]= \int_{-\infty}^{\infty} \frac{dg(x, \mu)}{d\mu} f(x, \mu) dx = \int_{-\infty}^{\infty} g(x, \mu) \frac{df(x)}{d\mu} dx$$
I mean, why not the chain rule inside the integral and only one function is partially differentiated?
It is easy to understand the rationale of it, but I am struggling with the proof...
 A: This question has such messy notation and mixes up so many different things that it is difficult to
figure out what exactly is being asked.
Let $g(x,\mu)$ denote an ordinary two-variable real-valued function of
two real variables. We will assume that the function is integrable with
respect to both variables. Let $X$ denote a normal random variable
with mean $\mu$ and variance $\sigma^2$. Thus, $X$ has a probability
density function given by
$$f_X(t) = f(t,\mu) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{1}{2}
\left(\frac{t-\mu}{\sigma}\right)^2\right),
~~-\infty < t < \infty.\tag{1}$$
Now, there is nothing random about the function $g(x,\mu)$ and so its
expected value is just $g(x,\mu)$ 1tself. Indeed the same notion,

the expected value of a constant is the constant itself

is a fundamental notion of probability theory (as well as of
real life).
More formally,
$$E[g(x,\mu)] = \int_{-\infty}^\infty g(x,\mu)f_X(t)\,\mathrm dt
= g(x,\mu)\int_{-\infty}^\infty f(t,\mu)\,\mathrm dt
= g(x,\mu). \tag{2}$$
and it is worth noting in passing that we would have obtained the
same answer even if we had used a different density function for
$X$, or indeed a discrete mass function, etc.  From $(2)$, we have
that 
$$\frac{\partial}{\partial\mu}E[g(x,\mu)]
= \frac{\partial}{\partial\mu}g(x,\mu)\tag{3}$$
and that's all there is to it. You could, if you like,
write $(3)$ as 
$$\frac{\partial}{\partial\mu}E[g(x,\mu)]
= \int_{-\infty}^{\infty}\frac{\partial}{\partial\mu}g(x,\mu)
f(t,\mu)\,\mathrm dt\tag{4}$$
which sort of looks like what you want to prove, but actually
isn't at all the same.

"But, but, but,..." you splutter indignantly, "All this is pure hooey.
Everybody, with the possible exception of Old Harry And His Old Aunt, knows that $(1)$ is incorrect: the density of $X$ is $f_X(x)$ and
not $f_X(t)$ as you have written in $(1$). Thus $(2)$ is not at all what
I know $E[g(x,\mu)]$ to be. It's got to be
$$E[g(x,\mu)] = \int_{-\infty}^\infty g(x,\mu)f(x,\mu)\,\mathrm dx
\tag{5}$$
the way I wrote it in my question."
All right, have it your way.. If $Y = g(X,\mu)$ is a random 
variable that is
a function of the random variable $X$, then the 
law of the unconscious statistician gives us that
$$E[g(X,\mu)] = \int_{-\infty}^\infty g(x,\mu)f(x,\mu)\,\mathrm dx.
\tag{6}$$
If you cannot see any difference between $(5)$ and $(6)$,
compare the 5th character in each equation; that is a fundamental
difference.
The equation $(6)$ can be differentiated on both sides
with respect to $\mu$ giving
\begin{align}
\frac{\partial}{\partial\mu}E[g(X,\mu)] &= \frac{\partial}{\partial\mu}\int_{-\infty}^\infty g(x,\mu)f(x,\mu)\,\mathrm dx\\
&=\int_{-\infty}^\infty \left(f(x,\mu)\frac{\partial}{\partial\mu}g(x,\mu)+g(x,\mu)\frac{\partial}{\partial\mu}f(x,\mu)\right)\,\mathrm dx\\
&= \int_{-\infty}^\infty f(x,\mu)\frac{\partial}{\partial\mu}g(x,\mu)\,\mathrm dx
+ \int_{-\infty}^\infty g(x,\mu)\frac{\partial}{\partial\mu}f(x,\mu)\,\mathrm dx\tag{7}
\end{align}
which more nearly resembles what you want to prove if you replace
that second $=$ sign in your question with a $+$ sign.
But note that 
$$\frac{\partial}{\partial\mu}f(x,\mu)
= \frac{x-\mu}{\sigma^2}f(x,\mu)$$
and so if $g(x,\mu)$ is an even function of $(x-\mu)$, then
the integrand in the second integral in $(7)$ is a odd function
of $(x-\mu)$, the value of that integral is $0$ and so
$$\frac{\partial}{\partial\mu}E[g(X,\mu)]
= \int_{-\infty}^\infty f(x,\mu)\frac{\partial}{\partial\mu}g(x,\mu)\,\mathrm dx$$
just the way you want it.
