# Markov chain with external input

Could anyone explain to me this Markov chain model? $$S_{k+1}= P(S_k+S_k^0).$$

Please allow me to give a link from the paper I was read this equation $$(6)$$ here https://drive.google.com/file/d/132FbOj-up5J4VO8ujj0wBI03aSQ28KJy/view?usp=sharing

I was actually reading this paper https://www.medrxiv.org/content/10.1101/2020.04.21.20073668v1.full.pdf but they refer to the above model which I didn't understand. Thanks for a bit of discussion and over-all idea about the model. Thanks a lot.

A simple Markov chain model multiplies an $$N\times N$$ square matrix of transition probabilities ($$P$$ in your question) by a state vector of height $$N$$ ($$S_k$$) in your question. The numbers in the new vector are probabilities (or relative frequencies) of the various states represented by each index. (I'm sure you know this, but sometimes $$P$$ is on the right or left, which affects the structure of that matrix--i.e. whether it's the rows or columns that must each sum to 1 because they represent probabilities. This difference is illustrated in the two papers.)

So in a simple Markov chain model, the only thing that produces a new state $$S_{k+1}$$ is the transition probability matrix $$P$$ operating on the old state $$S_k$$. We multiply the the square matrix $$P$$ and the column vector $$S_k$$ to produce the new state vector $$S_{k+1}$$.

In the model you are asking about, there is also an external source of change at time $$k$$, a vector of the same size as $$S_k$$, named $$S^0_k$$. This represents changes to the values represented in $$S_k$$ from some other source. Based on a remark two lines under the equation, it looks like this represents newly infected people to be added in to the vector of states of previously infected people, but I have not read the article closely.

Once the vectors $$S_k$$ and $$S^0_k$$ are added together, the next step proceeds just as in the simple Markov model: $$P$$ is multiplied by the resulting column vector.

EDIT:

Here is a concrete illustration. It's my version of the infection example from the second article that Miss Q linked to. Consider a disease where everyone who gets infected gets sick (not Covid-19). Suppose we want to understand how quickly reported numbers of people who are infected go from being sick, to recovered, or instead to having died. For any day $$k$$, we can represent can represent these numbers in a 3-element column vector $$S_k$$ representing numbers of people in states sick, recovered, dead. I'll put the number of sick people in the first, top element, and the number of dead people in the bottom, third element. (The second article uses row vectors, but the question used the notation of the first article, using column vectors.) I'm leaving out the $$S_k^0$$ part for the moment.

If we know how likely it is that sick people remain sick from day to day, or recover, or instead die, we can represent that with a $$3\times 3$$ transition matrix $$P$$. The elements of the matrix are probabilities, and each column sums to 1. The multiplication $$S_{k+1}=PS_k$$ calculates how many people in each of the three states transition to the other states. By the way, the second and third columns of $$P$$ will be

$$\begin{pmatrix} 0 \cr 1 \cr 0 \end{pmatrix} \mbox{ and } \begin{pmatrix} 0 \cr 0 \cr 1 \end{pmatrix}$$

respectively, on the assumption that recovered people remain recovered, and the dead remain dead.

One of the things you may have noticed is that in the preceding model, there is no way way to model new people getting infected. We start with a vector $$S_0$$ of infected people, and then track how fast they recover or die, and eventually, there are no sick people left. That's unrealistic! This is what the vector $$S_k^0$$ is for. It represents the number of newly reported infected people. Again, it is a 3-element column vector, with newly reported people who are either sick, recovered, or dead from the infection. On each day $$k$$, we collect the numbers of newly reported people in these three states, put these numbers in a column vector, and add the new numbers to the old data for people in the three states:

$$S_k + S_k^0$$

This gives us an updated number of people in the three states sick, recovered, dead. This is now the number we should multiply by the transition matrix to calculate the number of sick, recovered, and dead on day $$k+1$$:

$$S_{k+1} = P(S_k + S_k^0)$$

(Note: As I mentioned, in the second linked article, on Covid-19, the authors represent the data using row vectors, and they use a transition matrix with in which rows sum to 1, multiplying in the other direction. They also use $$t$$ instead of $$k$$, and leave out the time index on the vector representing newly reported cases. This resulting equation looks like this: $$S_{t+1}=(S_t+S^0)P$$. However, what is being calculated is the same.)

(By the way, for future reference, one should provide more background in the question, explaining exactly where you got stuck, rather than asking someone else to get extract of the relevant background information from external sources. This will help you think through the question, and help others direct their answers to whatever aspect of the problem is important to you.)

• I didn't understand your explanation. Thanks – Miss Q May 24 at 9:15
• What didn't you understand? You might want to add more information the posted question. – Mars May 25 at 1:35
• I didn't understand the $S_k^0$ part, what it is doing in each step and why I have to multiply with the transition matrix $P$ and then add – miosaki May 28 at 10:59
• @miosaki, I added a new section illustrating the ideas with a disease example inspired by one of the linked articles. I think it explains what the $S_k^0$ is doing. I know that what I added is a bit long, but I was trying to make the ideas clear for anyone. Let me know if it's not clear. – Mars May 28 at 15:02
• @miosaki, it looks like this question is copied from your question. Perhaps I should post my answer to your question instead. – Mars May 28 at 16:27