Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I believe the most restrictive assumption we can place on a series of observations is that they are iid.

It is possible to relax these assumptions. For example relaxing the independent distribution results in independent heterogeneously distributed random variables. In other words the distribution of each random variables can itself vary. What does this mean? There must be some importance or we would not bother to specify the properties of the distribution. Can the distribution depend on preceding observations or would this break the independent observation restriction that we still have in place?

Can anyone think of any intuitive examples to help explain this (in the same way that tossing a fair coin or rolling a fair die are good examples of iid observations) and any important statistical concepts that arise out of relaxing this assumption? My degree is not in statistics although I am quite interested in this subject. Thank you all so much.

share|improve this question
    
"relaxing the independent distribution results in independent heterogeneously..." Perhaps you mean "relaxing the identical ..." ? –  leonbloy Aug 29 '13 at 14:20

2 Answers 2

Relaxing the assumptions doesn't necessarily mean that you go from assuming a normal variable in period $t$, and a binomial in period $t+1$, or something similarly drastic. You are typically dealing with a parametric probability distribution; eg $\mathcal{N}(\mu, \sigma)$, which you would change to $\mathcal{N}(\mu_t, \sigma_t)$, or also, say, to something like $\mathcal{N}(\mu, \sigma_{f(x_0, ..., x_{t-1})})$, among many other variations. The last case is essentially what is done in models very commonly used in financial time series modeling, so-called ARCH models and their variants (c., e.g., http://en.wikipedia.org/wiki/Autoregressive_conditional_heteroskedasticity, or http://www.amazon.com/Time-Analysis-James-Douglas-Hamilton/dp/0691042896). Why this is useful depends on the case. When you look at historic time series, and sticking to this last example, you notice that the variance of certain time series tends to change over time. Hence, such a model would fit the data better than an iid assumption.

You have all kind of variations of the basic iid assumption that is originally driven from trying to fit real-world data better. To mention another example, if you have a time series in which crashes or booms occur (be that the stock market or simply a country's relative development), you might look at time-series which inherently will have discontinuous jumps, such as so-called regime-switch models (eg., http://en.wikipedia.org/wiki/SETAR_(model)).

All these models are considered useful, are widely employed, and well-investigated; so it should be easy to follow a google trail. If you were looking for book recommendations, I enjoyed the above-linked book by Hamilton which, when I was in grad school, was the standard (but this is over ten years ago). It is non-measure-theoretic, but rigorous, and was then current with research (I don't know if Hamilton kept updating it after).

share|improve this answer

Some parts of your question are not clear, but if you want examples in statistics in which weaker assumptions than i.i.d. can bear fruit, here are two:

  • The weak law of large numbers says that if $X_1,X_2,X_3,\ldots$ are uncorrelated (a weaker assumption than independence) random variables each with expected value $\mu$ and variance $\sigma^2$ (a weaker assumption that indentical distribution) then $$ \forall\varepsilon>0\ \lim_{n\to\infty} \Pr\left(\left|\frac{X_1+\cdots+X_n}{n}-\mu\right| < \varepsilon \right) = 1. $$

  • The Guass--Markov theorem considers a model $Y=X\beta+\varepsilon$ where $X\in\mathbb R^{n\times p}$ with $n\gg p$, $\beta\in\mathbb R^{p\times 1}$, $\varepsilon\in\mathbb R^{n\times 1}$ and so $Y\in\mathbb R^{n\times 1}$, and $X$ and $Y$ are observable data, $\beta$ unobservable and fixed, and $\varepsilon$ is unobservable and random. "Fixed" means not random; and "random" means something that changes if a new sample is taken. Thus $\varepsilon$ and hence $Y$ change if a new sample is taken, which the observable $X$ and the unobservable $\beta$ remain the same.

    One assumes the entries in $\varepsilon$ have expected value $0$ and all have the same variance $\sigma^2<\infty$ (a weaker assumption that indentical distribution) and they are uncorrelated (a weaker assumption than independence).

    The conclusion is that the best linear unbiased estimator (B.L.U.E.) of $\beta$ is $\hat\beta=(X^TX)^{-1}X^T Y$, the least-squares estimator. ("Linear" means linear as a function of $Y$.)

share|improve this answer
    
Can the person who down-voted this explain? –  Michael Hardy May 24 at 23:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.