# Notation i.i.d sample

I am learning measure theory and sometimes I am not sure if I am using the correct notations, especially with respect to distributions of random variables. In the following I try to formulate the estimation of a distribution from an i.i.d. sample.

Let $(\Omega,\mathcal{A},P)$ be a probability space and $(E,\mathcal{E})$, $(E_1,\mathcal{E}_1),\dotsc,(E_n,\mathcal{E}_n)$ be measurable spaces. The random variable $X\colon \Omega\to E$ is distributed according to $P_X$.

Let $X_i\colon \Omega\to E_i$ also be random variables and define a sample as $Z=(X_1,\dotsc,X_n)\colon \Omega\to E_1 \times \cdots \times E_n$ as a random varible with measurable space $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i)$. If the samples are independent, then $P_Z = P_{X_1} \otimes \cdots \otimes P_{X_n}$.

How do I correctly say that I am interested in the estimation of $P_X$, but have only access to independent observations from this distribution? Is in this case

• $P_Z = P_{X} \otimes \cdots \otimes P_{X}$
• $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i) = (\times_{i=1}^n E, \otimes_{i=1}^n \mathcal{E})$?

Furthermore, let $Z=z=(x_1,\dotsc,x_n)$ be a realization of the sample. Given the empirical distribution $$P_n(E) = n^{-1}\sum_{i=1}^{n} \mathbb{1}_E \circ X_i(\omega), \quad E\in\mathcal{E}, \omega \in \Omega$$ then $$P_Z\bigg( \Big\{\lim_{n\to\infty} P_n(E) = P_X(E) \Big\} \bigg) = 1, \quad E \in \mathcal{E} \quad \text{or}\\ P\bigg( \Big\{\lim_{n\to\infty} P_n(E)(\omega) = P_X(E) \Big\} \bigg) = 1, \quad E \in \mathcal{E}, \omega \in \Omega$$

Is the notation above formally correct?

• The proper notation is $$P\left(\lim_{n\to\infty} P_n(E) = P_X(E) \right) = 1$$ or alternatively, $$P\left( \left\{\omega\in\Omega\mid\lim_{n\to\infty} P_n(E)(\omega) = P_X(E) \right\} \right) = 1.$$ But maybe what you mean is the stronger statement that $$P\left(\forall E\in\mathcal E,\ \lim_{n\to\infty} P_n(E) = P_X(E) \right) = 1\ldots$$ Anyway, $P_Z$ in your first suggestion is incorrect since $P_Z$ is a measure on the target space $E^n$ and not on the source set $\Omega$ (and for which $n$ would one consider $Z$ anyway?), and $\omega$ in your second suggestion is incorrect as well.
– Did
Commented May 18, 2016 at 11:50
• @Did Thx. If you make your comment an answer I can mark it solved. It would be great if you could additionally state that my other definitions are also (in)correct. Commented May 18, 2016 at 12:54
• Before "Furthermore", your text is correct. Re an answer, why not write and post one yourself?
– Did
Commented May 18, 2016 at 16:38
• I could, but you did the work and hence deserve the reputation. I see you don't need it, but I thought it would be antisocial to "steal" the rep. Commented May 18, 2016 at 16:45
• Ego te absolvo... :-) Just post.
– Did
Commented May 18, 2016 at 16:48

Let $(\Omega,\mathcal{A},P)$ be a probability space and $(E,\mathcal{E})$, $(E_1,\mathcal{E}_1),\dotsc,(E_n,\mathcal{E}_n)$ be measurable spaces. The random variable $X\colon \Omega\to E$ is distributed according to $P_X$.
Let $X_i\colon \Omega\to E_i$ also be random variables and define a sample as $Z=(X_1,\dotsc,X_n)\colon \Omega\to E_1 \times \cdots \times E_n$ as a random varible with measurable space $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i)$. If the samples are independent, then $P_Z = P_{X_1} \otimes \cdots \otimes P_{X_n}$.
• $P_Z = P_{X} \otimes \cdots \otimes P_{X}$
• $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i) = (\times_{i=1}^n E, \otimes_{i=1}^n \mathcal{E})$
Furthermore, let $Z=z=(x_1,\dotsc,x_n)$ be a realization of the sample. Given the empirical distribution $$P_n(E) = n^{-1}\sum_{i=1}^{n} \mathbb{1}_E \circ X_i(\omega), \quad E\in\mathcal{E}, \omega \in \Omega$$ then $$P\bigg( \Big\{\omega \in \Omega \mid \lim_{n\to\infty} P_n(E)(\omega) = P_X(E) \Big\} \bigg) = 1$$ or alternatively, $$P\bigg( \lim_{n\to\infty} P_n(E) = P_X(E) \bigg) = 1$$