Is $E(f(x))$ always a constant? I'm a little confused about the definition of expectation of a function (eg not a random variable). Take a real valued function $f(x)$ which is non-zero everywhere, like $x^2+1$ maybe, and assume $p(x)> 0$ for any $x$. The expectation is defined as $$E(f(x)) = \int_{D(x)} f(x) p(x) dx.$$ $D(x)$ is the domain of $x$. Is this necessarily a constant, or can it be another functions? I think it can't be a constant like $E(f) = c$, otherwise we have the contradiction
$$\frac{d}{dx}\int_{D(x)} f(x) p(x) dx = \frac{d}{dx}  E(f(x)) \iff f(x)p(x) =0.$$
Is my understanding of this integral wrong, or have I made an error in my math? Or, is $E(f(x))$  going to be another function? Everything I have read says the expectation is going to typically be a single, scalar number, and this makes sense to me since the integral will return a real number. But maybe I am not correct about this definition, so there is no contradiction. Thank you for your help, and if you could cite a useful resource about it that would be great too.
 A: Usually we use uppercase $X$ for random variables and lowercase $x$ for real numbers. If $X$ is a random variable with probability density function $p(x)$ (notice how $p(x)$ is a function of the deterministic lowercase $x$), and $f$ is a function, then $\mathbb E[f(X)]=\int_{-\infty}^\infty f(x)\,p(x)dx$. This is always a constant, since it is telling us the average value of $f(X)$, which is a random quantity.
I should also say, we almost never have occasion to talk about the "expectation of a function," but if $X$ is a random variable, we can make more random variables by taking functions $f(X)$ of that random variable. Then we can ask about the means of these new random variables by using the expectation operator.

Edit:
I think a careful answer to your question needs to address the following point.
We need to be clear about what "taking expected value" means. In the context of probability theory, we can only take expected values of random variables, which we typically denote with uppercase letters, instead of lowercase letters—which we use for what are called deterministic variables.
So suppose we have a random variable $X$, which takes its values in the real numbers. Two examples, one specific and one general, would be

*

*$U$ takes values in the interval $[0,1]$, with the uniform distribution. Simply put, the probability that $U$ lies in any interval contained in $[0,1]$ is just the length of the interval.

*$X$ is a random variable taking values in $\mathbb R$ with a probability density function $p$. This means that to calculate the probability that $X$ lies in any interval contained in $\mathbb R$, we form the integral $$\mathrm{Prob}(a \le X \le b) = \int_a^b p(x)dx.$$
Now, if we are dealing with a random variable $X$, perhaps the most natural question we can ask about it is "What is the average value of $X$?" Let us call this number $\mathbb E[X]$ ($\mathbb E$ is for "expectation," as this quantity is also called the "expected" value, though this is somewhat of a misnomer, since a random variable need not ever equal its average value!). If we picked another random variable $X'$ with a distribution that was different from that of $X$, then $\mathbb E[X']\ne \mathbb E[X]$, in general, so the number we get when plugging in different random variables into $\mathbb E[\cdot]$ is different, but for a fixed random variable $X$, $\mathbb E[X]$ is a real number, and it represents the average value of $X$.
An interesting thing to do with random variables is to make new random variables with them. Say we are thinking about a uniformly random number $U$ in the interval $[0,1]$. Then we can make a new random number by squaring $U$. If $f(u) = u^2$ is the function that squares a real number, then let's call $X = f(U) = U^2$. Now we can ask, "What is the average value of $X$?" Because $U$ has a probability density function $p(u) = 1$ for $0\le u \le 1$, it is a special case of a theorem that is proved in a course touching on probability theory that $$\mathbb E[f(U)] = \mathbb E[U^2] = \int_0^1 u^2\,p(u)du = \int_0^1 u^2\,du = 1/3$$
In any case, I hope it is clear by now what $\mathbb E[X]$ means, and as a special case, what $\mathbb E[f(X)]$ means if $X$ is a random real number, and $f$ is a function. This is always a constant for a fixed random variable $X$.

Second edit: $\mathbb E [f(U)]=\int_{-\infty}^\infty f(u) p(u) du$ is not a function of $u$ because $\int_{-\infty}^\infty f(u) p(u) du$ is an integral with constant bounds of integration. If instead we were looking at $\int_{-\infty}^xf(u)p(u)du$, then this is a nonconstant function of $x$, and we could take $\frac{d}{dx}$ of this, and we would get $f(x)p(x)$. But then what we would be evaluating is $\frac{d}{dx} \mathbb E[f(U) ; U \le x]$, and not $\frac{d}{dx} \mathbb E[f(U)]$—which is just $0$. In words, $\mathbb E[f(U);U\le x]$ is the expected value of $f(U)$, on the event that the random values of $U$ are less than or equal to $x$. So $\mathbb E[f(U) ; U\le x]$ is a function of $x$ which is not constant in general because we get a different expected value if we change the parameter $x$, but this is a separate quantity from $\mathbb E[f(U)]$, and formally $\mathbb E[f(U)]$ equals $\mathbb E[f(U);U\le \infty]$.
A: If $X$ is a continuous random variable with density $p_X(x)$ defined on some support $D$ then the expected value of $f(X)$ (for some function $f$) is
$$
\mathsf Ef(X)=\int_D f(x) p_X(x)\,\mathrm dx,
$$
which is always a constant.
As a simple example, suppose $X\sim\operatorname{Uniform}(0,1)$ is a uniform random variable with support $D=(0,1)$ and choose $f(x)=x^2$. Then,
$$
\mathsf Ef(X)=\int_0^1 x^2\,\mathrm dx=\frac{1}{3}.
$$
A: Suppose we have $f: X\to Y$, then $E(f)=\int_X f(x)p(x)dx\in Y$
In your case, $Y=\Bbb R$
