# Exchange of Gradient and Expectation

Let $f : \mathbb{R}^n \to \mathbb{R}^k$ and let $\nabla$ be the gradient operator.

Under, what condition can we interchange \begin{align} \nabla_u E[f(X+u)]= E[ \nabla f(X+u)] \end{align}

I am well aware of the condition for the one dimensional case. See for example https://en.wikipedia.org/wiki/Leibniz_integral_rule

• In \begin{align} \nabla E[f(X)] \end{align}, with respect to which variables you are willing to take the derivative?
– Med
Commented Mar 20, 2017 at 21:28
• @Med There is only one variable in there $X$.
– Boby
Commented Mar 20, 2017 at 21:30
• $E(f[X])$ is a vector, with numbers as its components. Then, I think the gradient is the vector $0$. Am I making a mistake?
– Med
Commented Mar 20, 2017 at 21:33
• @Med I think you are right, but what is the gradient of a vector field anyway? Honestly, for me, the question doesn't even make sense. (Not that this means anything at all: I don't know too many things.) If someone has the time to provide an explanation (maybe even an answer), I'd like to ask that person to tag me, I'd love to read it. Commented Mar 20, 2017 at 22:25
• As pointed out above this does not make much sense. If $X$ is a random variable then $E[X]$ is just a number so there is no free variable to take the gradient with respect to so $\nabla E[f(X)] = 0$. The equality holds only if $E[\nabla f(X)] = 0$. Do you perhaps mean something else. You say you "are well aware of the condition for the 1D case"? Can you state it? Commented Mar 20, 2017 at 22:36

By analogy with the 1 dimensional case, I think what you really meant to ask is whether something like the following is true. $$\nabla_t E[f(X,t)]\overset{?}{=}E[\nabla_t f(X,t)]$$ In effect we are passing the derivative with respect to $t$ through the expectation with respect to $x$.

This interchange of differentiation and integration is valid under the exact same conditions as for the one dimensional case. Indeed, such an equality is equivalent to the 1 dimensional interchange holding for each component of the vector $t=(t_1,\ldots,t_n)$, that is, $$\frac{\partial E[f(X,t]}{\partial t_i}=E\frac{\partial f}{\partial t_i}(X,t),\qquad i=1,\ldots,n.$$

• Nice way to avoid answering the question, but is there any chance that you will answer directly to the question and explain the conditions?
– user168764
Commented Jul 26, 2020 at 19:45
• Well... the expected value in question is an integral with respect to a measure of total measure 1, so the paragraph of the cited Wikipedia article that addresses Measure theoretical statements applies. Explicitly: if the derivative of $f$ exists and there is a random variable $B$ with finite expected value and such that $|\nabla f(X\omega + u)| \leq B(\omega)$ for all $\omega \in \Omega$ and all $u$ in a neighborhood of $t$, $\nabla_u E(f(X + u)) \Big|_{u=t} = E( \nabla f(X+t) )$. A simple case is the case of bounded $\nabla f$ , because one can take $B=$ a large constant. Commented Nov 30, 2020 at 9:14