8
$\begingroup$

A story currently in the U.S. news is that an organization has (in)conveniently had several specific hard disk drives fail within the same short period of time. The question is what is the likelihood that this would happen? I would imagine that it can be determined quantitatively to be very unlikely, and I would like to know if a simple analysis is sufficient to come to that conclusion, or is it necessarily more complicated.

We can start with some assumptions, all of which can be challenged.

  1. A hard drive fails with an exponential probability density function (pdf) $p(t)= \lambda e^{- \lambda t}$, where $\lambda$ is the reciprocal of the MTBF (mean time between failures).
  2. All hard drives have an MTBF of 500,000 hours and operate under typical conditions.
  3. The hard drive failures are independent. (They run on different computers.There is no systematic relationship that would result in a dependency between these failures and any other commonly shared event or condition.)
  4. The failures are physical (not systematic, such as software induced)
  5. The organization operates 100,000 hard drives like these simultaneously.

While investigators argue about who has the better credentials relevant to the question, I believe it's amenable to a straightforward analysis, as follows:

An upper bound on $P_{1}(T)$, the probability of a single failure in time T, can be calculated from integrating the pdf on the interval $[0, T]$, where the pdf is maximum. The probability distribution $$P(t) = 1-e^{- \lambda t}$$ can be used to calculate $P(t=T)$.

The probability $P_{N}$ of N specific hard drives failing in that time interval is $P_{N}(T)=(P_{1}(T))^N$.

The facts in the actual investigation are not totally clear, but it appears that we are talking about 6 specific hard drives failing in a 1 week (168 hours). This leads to $$P_{1}(168)=1-e^{- 168/500,000}=3.36 \times 10^{-4}$$ and $$P_{6}(168)=1.44 \times 10^{-21}$$

This is so incredibly unlikely that I would try modifying my assumptions. First, if the time interval is 13 weeks, then $P_{6}(13*168)=6.85 \times 10^{-15}$. Still incredibly unlikely.

Even reducing the MTBF to 10,000 leaves us with $P_{6}(13*168)=5.7 \times 10^{-5}$ or nearly a one in a million chance.

One assumption that I didn't use was assumption number 5, that there are 100,000 hard drives within the organization. This is where the lies, damn lies and statistics creep in. But I think it's safe to say that this is irrelevant, given the other assumptions and that we are talking about specific hard drives.

Based on this analysis, calculating the probability that N specific hard drives would fail in an interval of time can be easily calculated. Have I made a mistake? Are there other factors that would have a significant effect on the result? If so, how?

$\endgroup$
  • 3
    $\begingroup$ The probability that those specific hard drives have failed is $1$ with hindsight. The probability that those specific hard drives would fail at similar times is small given the assumptions of your model, but the probability that any six of the 100,000 would fail at similar times is much higher, and it is wrong to consider those six in isolation with the benefit of hindsight: otherwise you can generalise and come to the conclusion that any reality is implausibly unlikely when in fact something has to happen. $\endgroup$ – Henry Jun 25 '14 at 16:17
  • 4
    $\begingroup$ There is actually a question of whether the hard drives did, in fact, fail. That assertion may fall under the category of damned lies. $\endgroup$ – Jim Jun 25 '14 at 16:23
  • $\begingroup$ @gnometorule Do you mean the news story that prompted the question, or an article that actually uses this type of analysis to make a conclusion? $\endgroup$ – apnorton Jun 25 '14 at 17:40
  • $\begingroup$ @anorton: an article with solid statistical analysis of how unlikely this event is. It kinda blows my mind what the IRS has been pulling recently, going back to flats in the Bahamas and such a while back (yes I realize that was no employee proper just...), with impunity. $\endgroup$ – gnometorule Jun 25 '14 at 17:46
3
$\begingroup$

Putting rough numbers in, with MTBF of $500,000$ hours, the chance a a given drive failing in a week is $\frac {168}{500,000}$. The average number that fail in a week is then $\frac {168\cdot 100,000}{500,000} \approx 34$. Presumably they have some backup process, which failed here, or we wouldn't hear about it. The issue is how many combinations of $6$ are there that will cause data loss. We would need to know how the backup system works.

$\endgroup$
  • $\begingroup$ What pdf are you assuming? If it were uniform over the period $[0, MTBF]$ then all drives would be expected to fail by the MTBF time, which isn't the case. $\endgroup$ – Jim Jun 25 '14 at 18:03
  • $\begingroup$ @Jim: for short periods compared to MTBF it is reasonable to consider a uniform distribution. Exponential failure will give the same answer because $1-\exp(-x) \approx x$ for $x\ll 1$ $\endgroup$ – Ross Millikan Jun 25 '14 at 18:05
  • $\begingroup$ @RossMilikan: I see. Then, since the question concerns 6 specific hard drives, it would be calculable what the likelihood is that they are part of that 34 arbitrary failures. $\endgroup$ – Jim Jun 25 '14 at 18:14
  • $\begingroup$ @Jim In what way are they specific? What makes the story different from "six of 100,000 hard drives failed, oops"? $\endgroup$ – Daniel Fischer Jun 25 '14 at 18:16
  • $\begingroup$ @DanielFischer: I am guessing what makes them specific is that some data was lost. This puts the independence assumption into question-maybe they were all in the same room which had an AC failure, or the same person didn't run the backup routine, or ... If you have this many hard drives, they fail all the time, and it should be a non-event. You restore from a backup and get on with life $\endgroup$ – Ross Millikan Jun 25 '14 at 18:21
0
$\begingroup$

As mentioned in the comments, we are not dealing with $6$ particular hard drives; rather, with some $6$ hard drives from a sample of $100,000$. So your $P_{N}(T)$ is actually binomially distributed with $10^{5}$ trials, and probability of success (rather, failure) $P_{1}(168)$. Therefore, the probability of $6$ hard drives failing in any given week is $$\binom{10^{5}}{6}(3.36\cdot10^{-4})^{6}(1-3.36\cdot10^{-4})^{99994}$$ for which I would probably use a normal approximation. But note that the mean here is $33.6$ failures per week, so there is probably some error or missing information in your story for this to be a significant event.

$\endgroup$
  • 1
    $\begingroup$ No, we are focused on 6 specific (particular) hard drives in this case. But this is where common (mis)understanding of statistics allows a detail like this to be obfuscated. But I think your calculation is right for any non-specific 6 out of the population failing. $\endgroup$ – Jim Jun 25 '14 at 18:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.