# Markov probability matrix and steady state solutions

I am currently doing a question and can seem to find where my problem lies with my final solution. The question is based on markov probability and steady state solutions, but the solution I get for the steady state the probability don't add to one, and I cant see where I have gone wrong wheather is a miss calculation or my understanding or Markov matrices is incorrect.

The question is quite long as one part leads on to the other I will display the question and my final solution for each section to save some white space.

Part 1

For this I used the standard equation of the form $$(M-b)\bar{x}=0$$ to find the eigenvalues, then subbed the eigenvalue back into the equation $$(M-b)\bar{x}=0$$ to find the eigenvectors.

To which my final results were.

$$\lambda _1=a+b,\:\:\bar{x}=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\ 1\end{pmatrix}$$

$$\lambda _2=a-b,\:\:\bar{x}=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\ -1\end{pmatrix}$$

part 2

For this I used the conditional probability to form the matrix

$$\begin{pmatrix}p&1-p\\ 1-p&p\end{pmatrix}$$

part 3

I first used the general equations for $$\lambda_1$$ and $$\lambda_2$$ to get the eigenvalues

$$\lambda_1=1$$ $$\lambda_2=2p-1$$

Using the same for the eigenvectors obtained from part 1. I then went on to from $$T$$

$$T=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1&0\\ 0&2p-1\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}$$

Part 4

using the above equation I formed the steady state solution

$$\begin{pmatrix}u_{n+1}\\ v_{n+1}\end{pmatrix}=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1^n&0\\ 0&\left(2p-1\right)^n\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}u_0\\ v_0\end{pmatrix}$$

as $$n \rightarrow \infty$$ then I got

$$\begin{pmatrix}u_{n+1}\\ v_{n+1}\end{pmatrix}=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1&0\\ 0&0\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}u_0\\ v_0\end{pmatrix}$$

by subbing in the condition for i)

$$\begin{pmatrix}u_{n+1}\\ v_{n+1}\end{pmatrix}=\frac{1}{4}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}1&0\\ 0&0\end{pmatrix}\begin{pmatrix}1&1\\ 1&-1\end{pmatrix}\begin{pmatrix}0.2\\ 0.8\end{pmatrix}$$

the state vector comes out to be $$\begin{pmatrix}0.25\\ 0.25\end{pmatrix}$$ which cannont be correct at the state vector should sum to 1.

Could someone maybe expand upon where I have gone wrong. I understand this is a long post but even if someone could point to the part of the question where my mistake is and just give a little explanation.

• In part (c), your $1/4$ should be $1/2$. – Fabio Somenzi May 4 at 19:56
• The arithmetic error in part (c) aside, why are you setting $(u_0,v_0)=(0.2,0.8)$ at the end? Those are two different values of $p$ for which you’re supposed to do the computation, not the initial state, which should represent whether or not he scored on the very first penalty of the season, i.e., it’s either $(1,0)^T$ or $(0,1)$. – amd May 4 at 22:39
• Ah okay i see the arthimatic error but, with regrads to 0.2, 0.8 if they are values of P but at time gose on P will become very small over a large number so each would become zero. Which is why I thought the question refered to the probabilty of the intial state. – james2018 May 5 at 8:47
• I reviewed my working but still am not sure then why 0.2 and 0.8 for p are necessary as, I stated above as n get large 2p-1 will tend to 0, so it going to be the same for each case. Is my T martirx incorrect. I have refered to other sites sosmath.com/matrix/markov/markov.html for other example but seem to have done what I have outlined above. – james2018 May 5 at 10:40