I have learned that, for an i.i.d. random process $\{X_k\}$, $k=1, 2, \ldots$, $$ \frac{1}{n}\sum_{k=1}^n{X_k}\longrightarrow EX_1 $$ in probability sense.

For a stationary random process $\{X_k\}$, $k=1, 2, \ldots$, $$ \frac{1}{n}\sum_{k=1}^n{X_k}\longrightarrow EX_1 $$ with probability $1$.

However, someone told me that for an i.i.d. random process, its sample mean converges to expectation value with probability $1$ as $n$ grows.

I am confused about the fact between WLLN and SLLN. Isn't there any problem that I admit that i.i.d. random processes are also having stationarity?

  • $\begingroup$ This proposition is true if $\operatorname{var}(X_1)<\infty$. However, if the distribution is $\displaystyle \left(\text{constant} \cdot \frac{dx}{1+x^2}\right)$ then it is false. $\qquad$ $\endgroup$ – Michael Hardy Oct 26 '16 at 0:46
  • $\begingroup$ @MichaelHardy Ah, thank you for supplement. I omitted finite variance. And, do you mean that the distribution refers to CDF of $X$?? $\endgroup$ – Danny_Kim Oct 26 '16 at 3:07
  • 1
    $\begingroup$ The density is $x\mapsto \dfrac{\text{constant}}{1+x^2}$; therefore the distribution is $\left( \text{constant} \cdot \dfrac{dx}{1+x^2}\right)$ and the cumulative distribution function is $x\mapsto \dfrac 1 \pi\cdot \arctan x. \qquad$ $\endgroup$ – Michael Hardy Oct 26 '16 at 3:14

A strict(strong)-sense stationary process $\{X_t\}$ is one whose joint distributions for any set of times $t_1,\ldots,t_k$, that $F_X(t_1,\ldots,t_k) = F_X(t_1+\tau,\ldots,t_k+\tau)$ for any $\tau$.

An i.i.d. process always satisfies this, since its joint distribution at any set of times is the same.

If you have an i.i.d. sequence of finite mean random variables, then the sample average converges to the mean with probability one (strong law of large numbers) and in probability (weak law of large numbers).

Not all iid processes have finite means though (e.g. a sequence of i.i.d. Cauchy RV's) so the LLN's don't hold. And not all stationary processes will satisfy some conditions for a law of large numbers (in whatever sense -- mean square, probability, almost surely, etc.).

  • $\begingroup$ Thank you. I will read some material on internet more based on your answer. $\endgroup$ – Danny_Kim Oct 26 '16 at 3:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.