3
$\begingroup$

I am learning measure theory and sometimes I am not sure if I am using the correct notations, especially with respect to distributions of random variables. In the following I try to formulate the estimation of a distribution from an i.i.d. sample.

Let $(\Omega,\mathcal{A},P)$ be a probability space and $(E,\mathcal{E})$, $(E_1,\mathcal{E}_1),\dotsc,(E_n,\mathcal{E}_n)$ be measurable spaces. The random variable $X\colon \Omega\to E$ is distributed according to $P_X$.

Let $X_i\colon \Omega\to E_i$ also be random variables and define a sample as $Z=(X_1,\dotsc,X_n)\colon \Omega\to E_1 \times \cdots \times E_n$ as a random varible with measurable space $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i)$. If the samples are independent, then $P_Z = P_{X_1} \otimes \cdots \otimes P_{X_n}$.

How do I correctly say that I am interested in the estimation of $P_X$, but have only access to independent observations from this distribution? Is in this case

  • $P_Z = P_{X} \otimes \cdots \otimes P_{X}$
  • $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i) = (\times_{i=1}^n E, \otimes_{i=1}^n \mathcal{E})$?

Furthermore, let $Z=z=(x_1,\dotsc,x_n)$ be a realization of the sample. Given the empirical distribution $$P_n(E) = n^{-1}\sum_{i=1}^{n} \mathbb{1}_E \circ X_i(\omega), \quad E\in\mathcal{E}, \omega \in \Omega$$ then $$ P_Z\bigg( \Big\{\lim_{n\to\infty} P_n(E) = P_X(E) \Big\} \bigg) = 1, \quad E \in \mathcal{E} \quad \text{or}\\ P\bigg( \Big\{\lim_{n\to\infty} P_n(E)(\omega) = P_X(E) \Big\} \bigg) = 1, \quad E \in \mathcal{E}, \omega \in \Omega $$

Is the notation above formally correct?

$\endgroup$
5
  • 1
    $\begingroup$ The proper notation is $$P\left(\lim_{n\to\infty} P_n(E) = P_X(E) \right) = 1$$ or alternatively, $$P\left( \left\{\omega\in\Omega\mid\lim_{n\to\infty} P_n(E)(\omega) = P_X(E) \right\} \right) = 1.$$ But maybe what you mean is the stronger statement that $$P\left(\forall E\in\mathcal E,\ \lim_{n\to\infty} P_n(E) = P_X(E) \right) = 1\ldots$$ Anyway, $P_Z$ in your first suggestion is incorrect since $P_Z$ is a measure on the target space $E^n$ and not on the source set $\Omega$ (and for which $n$ would one consider $Z$ anyway?), and $\omega$ in your second suggestion is incorrect as well. $\endgroup$
    – Did
    Commented May 18, 2016 at 11:50
  • $\begingroup$ @Did Thx. If you make your comment an answer I can mark it solved. It would be great if you could additionally state that my other definitions are also (in)correct. $\endgroup$ Commented May 18, 2016 at 12:54
  • $\begingroup$ Before "Furthermore", your text is correct. Re an answer, why not write and post one yourself? $\endgroup$
    – Did
    Commented May 18, 2016 at 16:38
  • $\begingroup$ I could, but you did the work and hence deserve the reputation. I see you don't need it, but I thought it would be antisocial to "steal" the rep. $\endgroup$ Commented May 18, 2016 at 16:45
  • $\begingroup$ Ego te absolvo... :-) Just post. $\endgroup$
    – Did
    Commented May 18, 2016 at 16:48

1 Answer 1

2
$\begingroup$

I post here the corrected example.

Let $(\Omega,\mathcal{A},P)$ be a probability space and $(E,\mathcal{E})$, $(E_1,\mathcal{E}_1),\dotsc,(E_n,\mathcal{E}_n)$ be measurable spaces. The random variable $X\colon \Omega\to E$ is distributed according to $P_X$.

Let $X_i\colon \Omega\to E_i$ also be random variables and define a sample as $Z=(X_1,\dotsc,X_n)\colon \Omega\to E_1 \times \cdots \times E_n$ as a random varible with measurable space $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i)$. If the samples are independent, then $P_Z = P_{X_1} \otimes \cdots \otimes P_{X_n}$.

In this case

  • $P_Z = P_{X} \otimes \cdots \otimes P_{X}$
  • $(\times_{i=1}^n E_i, \otimes_{i=1}^n \mathcal{E}_i) = (\times_{i=1}^n E, \otimes_{i=1}^n \mathcal{E})$

Furthermore, let $Z=z=(x_1,\dotsc,x_n)$ be a realization of the sample. Given the empirical distribution $$P_n(E) = n^{-1}\sum_{i=1}^{n} \mathbb{1}_E \circ X_i(\omega), \quad E\in\mathcal{E}, \omega \in \Omega$$ then $$ P\bigg( \Big\{\omega \in \Omega \mid \lim_{n\to\infty} P_n(E)(\omega) = P_X(E) \Big\} \bigg) = 1 $$ or alternatively, $$ P\bigg( \lim_{n\to\infty} P_n(E) = P_X(E) \bigg) = 1 $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .