If there is some sample $X^n=(x_1,x_2,\dots,x_n)$, do we consider the elements of this sample $x_i$ independent and identically-distributed realizations of the same random variable $X$ or are they all realizations of different independent and identically distributed random variables $X_1,X_2,...,X_n$ (observation $x_1$ is the realization of a random variable $X_1$, observation $x_2$ is the realization of a random variable $X_2$ etc.)?

I hope this makes sense.

  • 1
    $\begingroup$ I do not see how this makes a difference to the distribution. But to the extent that $X_1$ is a random variable rather than an observation $x_1$ of a random variable, I suspect that the second description is more applicable $\endgroup$
    – Henry
    Nov 29 '19 at 12:57
  • 1
    $\begingroup$ Just in case you don't get an answer that makes this concepts clear to you, I would suggest checking the difference between random variate (or realization) and random variable. $\endgroup$ Nov 29 '19 at 13:30
  • 3
    $\begingroup$ What exactly do you mean with "realization of random variable"? A random variable $X$ is not more than a function with specific properties. For every outcome $\omega$ in sample space $\Omega$ a value $X(\omega)$ "shows up". Is that the realization that you are talking about? If so then realize that there is only one such value if there is only one random variable $X$. $\endgroup$
    – drhab
    Nov 29 '19 at 13:42
  • 1
    $\begingroup$ The key sentence is "The value of the random variable (that is, the function) $X$ at a point $\omega \in \Omega,$ i.e. $ x=X(\omega )$ is called a realization of $X$." This would be for a single random variable. In your post $X^n=(x_1,x_2,...,x_n)$ is a realization of $n$ independent and identically distributed random variables, or the random vector $X=\begin{bmatrix}X_1,X_2,\cdots,X_n\end{bmatrix}^\top.$ $\endgroup$ Nov 29 '19 at 13:53
  • 1
    $\begingroup$ @AntoniParellada thank you for your helpful comments $\endgroup$ Nov 29 '19 at 15:36

You may think about $(x_1,\ldots,x_n)$ as a realization of $n$ independent copies of $X$. Basically, there is a probability space $(\Omega,\mathcal{F},\mathsf{P})$ in the background so that $(x_1,\ldots,x_n)=(X_1(\omega)\ldots,X_n(\omega))$ for some $\omega\in\Omega$, which is chosen randomly according to $\mathsf{P}$. Then the statement "independent and identically-distributed realizations of the same random variable" doesn't make sense. Although, sometimes $(x_1,\ldots,x_n)$ is referred to as a random sample from a particular distribution (e.g. $F_X$).

  • $\begingroup$ thank you for your answer, very helpful. So if those copies of $X$ are separate entities for each $x_i$ in $(x_1,...,x_n)$, can you comment on why "sometimes $(x_1,...,x_n)$ is referred to as a random sample from a particular distribution"? I mean it makes sense: you have a distribution and you drag values from it. But on the other hand, a particular distribution describes probabilities of values for a particular random variable. Do those "copies of $X$" shrink to one entity when people talk about samples from a distribution? $\endgroup$ Nov 29 '19 at 15:28
  • $\begingroup$ "Drawing a random sample from a distribution" means obtaining a realization of i.i.d. random variables having that distribution. I think that the origin of that phrase may be related to the inverse transform sampling. $\endgroup$
    – d.k.o.
    Nov 29 '19 at 15:35
  • $\begingroup$ I got one question: Imagine that we have $n$ uncorrelated (not independent) copies of $X$. Now imagine we have one realization $(x_1,...,x_n)$ of these $n$ copies. Can we use this realization $(x_1,...,x_n)$ to estimate the mean of $X$? In other words, can we use samples from uncorrelated identically distributed random variables to estimate the mean (for example, using the sample mean)? $\endgroup$
    – Veljko
    Feb 6 '20 at 20:04
  • $\begingroup$ @Veljko Sure. The smaple average of pairwise uncorrelated r.v.s. converges to the mean of these r.v.s. (simple application of Chebyshev's inequality). $\endgroup$
    – d.k.o.
    Feb 6 '20 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.