Let's assume we have a lightbulb with a maximum lifespan 4 years. We are asked to create a transition matrix (Markov chain theory) for the bulb. The bulb is checked once a year and if it's found that the bulb does not work it is replaced by a new one.

We know that the probabilities of failure during the 4 year period are: 0.2, 0.4, 0.3 and 0.1. After four years, the lightbulb is replaced with probability 1. So we have the following 4 states:

  • $S_0$ - the lightbulb is new
  • $S_1$ - the lightbulb is 1 year old
  • $S_2$ - the lightbulb is 2 years old
  • $S_3$ - the lightbulb is 3 years old

How to create the probability transition matrix? For the first year, it's clear. The bulb goes dead with probability $p_{0,0} = 0.2$ and it keeps working with probability $p_{0,1} = 0.8$. But I am not sure how to calculate to the following years. In the materials for my course, I found the following calculation:

$$ p_{2,1} = \frac{0.4}{0.8} \, , p_{2,3} = \frac{0.3+0.1}{0.8} = 0.5 \, , p_{3,1} = \frac{0.3}{0.4} = 0.75 \, , p_{3,4} = \frac{0.1}{0.4} = 0.25 $$

So the probability transition matrix is:

\begin{bmatrix} 0.2&0.8&0&0\\0.5&0&0.5&0\\0.75&0&0&0.25\\1&0&0&0\end{bmatrix}

Is this correct? I fail to see why $p_{2,3}$ uses the probability of failure in the last year.

  • 1
    $\begingroup$ The matrix is right. It's Bayes' theorem. $\endgroup$ – Oolong milk tea May 11 '19 at 11:50
  • $\begingroup$ Does the second row represent the transition probabilities from $S_1$ to $S_0$, $S_1$,..,$S_3$, respectively? $\endgroup$ – John Douma May 11 '19 at 12:01
  • $\begingroup$ @JohnDouma Yes, it does. $\endgroup$ – Jiří Pešík May 11 '19 at 12:04
  • $\begingroup$ Then shouldn't those $0.5$s be replaced by $0.4$ and $0.6$? $\endgroup$ – John Douma May 11 '19 at 12:05
  • $\begingroup$ @JohnDouma The numbers are results of $p_{2,1} = 0.5$ and $p_{2,3} = 0.5$ given above the matrix. The point is I am not sure if they're right or not. $\endgroup$ – Jiří Pešík May 11 '19 at 12:08

The probabilities known, summing to $1$, are the probability at birth of failing during the 1st, 2nd, 3d or 4th year. Upon failure, the bulb is replaced with a new one, which thus has the same probabilities as above.
We have therefore the following scheme.


So the probability (at birth) $P_2$ to fail in the 2nd year (not before, and not after) will be given by the probability to survive for the first year times the probability $p_2$ to fail exactly in the 2nd year (given that it survived the first). And analogously for the others, i.e. $$ \eqalign{ & p_{\,1} = 0.2 \cr & \left( {1 - p_{\,1} } \right)p_{\,2} = 0.8 \cdot p_{\,2} = P_{\,2} = 0.4\quad \Rightarrow \quad p_{\,2} = 0.5 \cr & \left( {1 - p_{\,1} } \right)\left( {1 - p_{\,2} } \right)p_{\,3} = P_{\,3} \quad \quad \Rightarrow \quad p_{\,3} = 0.75 \cr & \left( {1 - p_{\,1} } \right)\left( {1 - p_{\,2} } \right)\left( {1 - p_{\,3} } \right)p_{\,4} = P_{\,4} \quad \Rightarrow \quad p_{\,4} = 1 \cr} $$

And $p_k,(1-p_k)$ are the entries of the matrix.


The key lies in interpreting the phrase “the probabilities of failure during the 4 year period” (was the original in English?). It’s pretty unlikely that these numbers represent the transition probabilities that you’re trying to construct. It would be a rather miraculous light bulb that gets more reliable the longer it’s in service. The numbers sum to $1$ and we know that the bulb only lasts four years maximum, so the likelier reading is that these numbers are the probabilities that when the bulb fails, it is in its $k$th year of service. That is, these four numbers are the probabilities that is it year $k$ or service given that the bulb has been found to have failed.

This reading is borne out by the computations in the course materials, which are applications of Bayes’ theorem (as pointed out by Oolong milk tea). For the transition matrix, you need the probabilities that the bulb is found to have failed given that it is year $k$ of service, which is just the sort of thing that Bayes’ theorem allows you to compute from the given data.

So, for example, the denominator of $p_{21}=0.8$ is the probability that the light bulb is one year old, i.e., the probability that it didn’t fail in its first year of service, which is simply $p_{12}=0.8$. The numerator is the probability of the bulb’s being a year old given that it failed the test, which is $0.4$ from the given probabilities, multipled by $1$ since it did fail the test. The computation of $p_{23}$ is a bit odd. It looks like the authors used $$\Pr(\text{test fails in third year} \cup \text{test fails in fourth year}) = 0.3+0.1$$ for the probability that the bulb passes when tested during the second year. One could’ve simply set $p_{23}=1-p_{21}$ since there are only two possible transitions from $S_1$. Fortunately, the two values agree.

  • $\begingroup$ The original article was not in English, however, it was described very briefly. Your interpretation makes sense to me. $\endgroup$ – Jiří Pešík May 12 '19 at 20:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.