10
$\begingroup$

To define outliers, why we cannot use: Lower Limit: Q1-1xIQR Upper Limit: Q3+1xIQR

OR

Lower Limit: Q1-2xIQR Upper Limit: Q3+2xIQR

$\endgroup$
8
$\begingroup$

By definition, 50% of all measurements are within $\pm0.5IQR$ of the median. Compare this - heuristically - with a normal distributions where 68% are within $\pm\sigma$, so in that case IQR would be slightly less than $\sigma$. Cutting at $\pm 1.5IQR$ is therefore somewhat comparable to cutting slightly below $\pm3\sigma$, which would declare about 1% of measurements outliers. This matches quite well with the habit of using "$3\sigma$" as a bound in many simple statistical tests. On the other hand, cutting at $\pm1IQR$ would be like cutting near $\pm 2\sigma$, making about 5% outliers - too many; and cutting at $\pm2IQR$ would be like cutting at $\pm4\sigma$, thus turning even many quite extreme measurements into non-outliers. So $\pm 1.5IQR$ is also what Goldilocks would choose.

$\endgroup$
  • 1
    $\begingroup$ I don't understand that " IQR would be slightly less than σ". Because one is IQR while another one is Natural distribution, how can you convert one side to another so that you can compare. Also, how did you calculate the 1% and 5%? $\endgroup$ – Fan Oct 10 '14 at 14:41
  • 2
    $\begingroup$ "By definition, 50% of all measurements are within ±0.5IQR of the median." This is true if the median is halfway between the first and third quartiles, but not necessarily otherwise. $\endgroup$ – Rahul Jun 21 '16 at 18:27
  • $\begingroup$ The phrasing $\pm 1.5 IQR$ seems to imply that something is an outlier if it's more than $1.5 IQR$ away from the median, which is not correct. $\endgroup$ – Richard Rast Jul 9 '18 at 13:11
4
$\begingroup$

We certainly CAN use whatever outlier bound we wish to use, but we will have to justify it eventually. In the not-so-recent past, it was typical to expect distributions to be Gaussian. With that assumption, ±1IQR is too exclusive, resulting in too MANY outliers, ±2IQR is too inclusive, resulting in too FEW outliers. ±1.5IQR is easy to remember, and is a reasonable compromise, under assumptions of Gaussianity.

However, for your distribution and expected outlier fraction, those assumptions may not be appropriate. Additionally, perhaps the definition of outlier is incorrect for your problem, and requires greater detail than just how it behaves within the bounds of a single metric?

$\endgroup$
3
$\begingroup$

The 3rd quartile (Q3) is positioned at .675 SD (std deviation, sigma) for a normal distribution. The IQR (Q3 - Q1) represents 2 x .675 SD = 1.35 SD. The outlier fence is determined by adding Q3 to 1.5 x IQR, i.e., .675 SD + 1.5 x 1.35 SD = 2.7 SD. This level would declare .7% of the measurements to be outliers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.