Stratified sampling

Question

Can stratified sampling be used to deal with the outliers that a data set might have? For the data set that I have, when trying to fit a statistical distribution and calculate a Value at Risk by using the Loss Distribution Approach, I find that the outliers( in my case defined based on the 3 standard deviation rule ) distort quite a lot the VAR values and the summary statistics. I read that in statistics one would eliminate the outliers but in this case if eliminated, we loose valuable information. Thus, I was wondering if stratified sampling could be used to deal with these outliers rather then eliminating them to make them more representative for the whole data set ! Many thanks!

Answer 1

The literature on outliers in survey sampling distinguishes (since Chambers, 1986) two cases of outliers:

• Case A) representative outliers (atypical observations but not necessarily errors; e.g. sample on turnover data for some companies, where one single company has a huge turnover compared to the others => atypical observation)
• Case B) non-representative outliers (errors of some kind)

What are our possible actions? In case B), the non-representative outlier does not contribute to answering our research question. Hence it is usually discarded from the analysis (or heavily weighted down such that it does not harm). In case A), we should not discard the atypical observations as it is part of the sample and it is supposed to carry some relevant information (i.e. not an error). Case A) is often encountered if the data are quite dispersed / heterogeneous. It may however be advisable to downweight this observation in order to limit its influence on the overall estimates (or summary statistics). Now, if we wish to limit the influence of outliers, we are faced with a tradeoff between variance and bias, which is easily seen from the following: If we limit the influence (or, equivalently, downweight the observation) of an extremely large observation too much, we incur [on the one hand] some bias (as the observation cannot contribute with its full “strength” although it is regarded correct). On the other hand, this strategy will provide us with smaller variance estimates (as the contribution of this atypically large observation is limited). And vice versa. There are some technical criteria that guide us how to resolve this tradeoff; however, in practice one typically relies on experience or some rules of thumb).

How does this relate to stratification? With respect to «stratification», we have to distin-guish:

• Case i) stratification in the context of “stratified sampling” prior to data collection (i.e. a sampling design that divides the population into strata and plans to draw samples from each stratum separately; perhaps using different sampling schemes in each stratum)

• Case ii) post-stratification after data collection (i.e. stratification of the collected data along certain stratification or grouping variables having observed all data points)

In Case i), our data consist of tuples $(y_{ij}, w_{ij})$, where $i=1,\ldots,n_i$ denotes the index of the $i$th unit (e.g. company) in the $j=1,\ldots,J$ stratum; $w_{ij}$ is the sampling weight [for ease of simplicity, I restrict attention to only one study variable]. In case ii), all weights are supposed to be equal since (by assumption) no complex sampling design (i.e. no stratification, no clustering, etc.) has been used; hence we can ignore the weights (for our purposes), and we regard $y_i$ as our data, $i=1,\ldots,n$.

Consider Case i). Here we have several options to limit the influence of outliers

• 1) use of robust statistical methods (e.g. outlier robust regression) instead of classical methods (robust methods typically downweight, loosely speaking, outliers in view of their values $y_{i}$. Giving an observation the weight = 0 is equivalent to discard it from the analysis)

• 2) reduce the sampling weight $w_{ij}$; this limits the contribution of the observation through the "sample-weighting channel" (for theoretical reasons sampling weights should be larger than or equal to one; however, in practice, weights smaller than one are used sometimes)

• 3) a mixture of the above two methods, i.e., a combination of downweighting the influence of the tuple $(y_{ij}, w_{ij})$

Consider Case ii) In general, post-stratification is inappropriate to deal with outliers. Suppose we divide the overall sample $s$ into $j=1,\ldots,J$ (disjoint) post-strata $s_j$, such that $s=\cup_{j=1}^J s_j$. Now, our outlier will reside in a particular $s_j$. When we are interested in overall sample estimates, the stratum containing our outlier will contribute to the estimates (except we discard the whole stratum). Therefore, you are recommended to use robust statistical methods instead.

My recommendations for your situation: Unless you have very, very good reasons, you should not discard observations from your analysis. It is better to keep atypical observations but to limit their influence, e.g. downweight them. You are highly recommended to use a formal, methodological approach to downweight / limit such observations (for reason of reproducibility). This can be achieved by robust statistical methods (e.g. outlier robust regression).

References

• Maronna, Martin, and Yohai (2006): Robust Statistics: Theory and Methods, John Wiley and Sons: Comment This is book gives a good and rather non-technical overview (compared with other books on robust statistics)

• Beaumont and Rivest (2009) Dealing with Outliers in Survey Data, in: Handbook of Statistics, ed. by Pfeffermann and Rao, Vol. 29A, chapter 11, Elsevier: Amsterdam Comment: this handbook article gives a good overview and provides a large number of references to dive deeper into the topic

• Chambers (1986) Outlier Robust Finite Population Estimation, Journal of the Ameri-can Statistical Association 81, p. 1063-1069: Comment see here if you are interested in the discussion of representative vs. non-representative outliers

Answer 2

If you have a strongly right-skewed population distribution, you should expect to get some observations that seem to be outliers. And I don't see how stratified sampling would be a 'cure' for this.

I simulated a sample of $n = 50$ observations from the exponential distribution with mean $\mu = 1.$ The average is $\bar X = 0.96$ and the SD is $s = 1.12.$ A boxplot is shown below. The $=$ symbol is at the mean and the $-$ is at $\bar X + 3s.$ By the '3 SD' rule, there are two outliers. By the more commonly used boxplot method, there are four outliers, plotted as circles.

For an exponential distribution, 'extreme' values of this nature are not surprising. If your data are the heights of people, I would not expect to see many outliers. If they are incomes, I would expect to see many. Depending on circumstances, maybe you should be trying to understand the 'outliers' rather than trying to avoid observing them.