Proof that Expectation of a Function with respect to the Empirical Distribution is the Average of the Function Evaluated at the Observed Points Can someone please provide the proof or point to any reference with detailed steps?
Question,
Why is the expectation of a function with respect to the empirical distribution, the average of the function evaluated at the observed points as shown below?
Defining the empirical distribution function as,
$$
  \hat{F}_n(t)=\frac{1}{n}\sum_{i=1}^n I_{[x_i,\infty)}(t) \, ,
$$
Why does it follow that?
$$
  \int_{-\infty}^\infty g(t)\,d\hat{F}_n(t) = \frac{1}{n} \sum_{i=1}^n g(x_i) \, .
$$
Related Question
I was unable to find this proved on the website or elsewhere. If very well known result, am happy to delete the question, once someone gives me a reference with detailed steps.
 A: Using the definition of $dF_n(t)$ provided above by @Did,
First we note a useful result and the required follows as shown below,
$$
\int_{0}^{K}dF_{n}\left(t\right)=\int_{0}^{K}\frac{1}{n}\sum_{i=1}^{n}\delta_{X_{i}}\left(t\right)dt=\frac{1}{n}\sum_{i=1}^{n}\boldsymbol{1_{\left\{ X_{i}\in\left[0,K\right]\right\} }}=\frac{1}{n}\sum_{i=1}^{n}\boldsymbol{1_{\left\{ X_{i}\leq K\right\} }}=F_{n}\left(K\right)
$$
$$
\left[\because dF_{n}=\frac{1}{n}\sum_{i=1}^{n}\delta_{X_{i}};\text{ Here, }\delta_{x}(A)=1_{A}(x)=\begin{cases}
0, & x\not\in A;\\
1, & x\in A.
\end{cases}\text{ is the Dirac measure}\right]
$$
More generally,
$$
\int_{0}^{K}g\left(t\right)dF_{n}\left(t\right)=\int_{0}^{K}\frac{1}{n}\sum_{i=1}^{n}g\left(t\right)\delta_{X_{i}}\left(t\right)dt=\frac{1}{n}\sum_{i=1}^{n}g\left(X_{i}\right)\boldsymbol{1_{\left\{X_{i}\leq K\right\} }}
$$
$$
\int_{0}^{\infty}g\left(t\right)dF_{n}\left(t\right)=\int_{0}^{\infty}\frac{1}{n}\sum_{i=1}^{n}g\left(t\right)\delta_{X_{i}}\left(t\right)dt=\frac{1}{n}\sum_{i=1}^{n}g\left(X_{i}\right)\boldsymbol{1_{\left\{ X_{i}\leq\infty\right\} }}=\frac{1}{n}\sum_{i=1}^{n}g\left(X_{i}\right)
$$
