6

The transition matrix reads: $P(X_n=0\to X_{n+1}=0)=\frac12$, and $P(X_n=0\to X_{n+1}=1)=\frac12$, and $P(X_n=0\to X_{n+1}=2)=0$. This is me reading off the first row. In the context, this means the probability of going from $0$ successive heads to $0$ successive heads is $\frac12$ which makes sense, since this is the probability of rolling a tails. The ...


5

Yes, this is a Markov chain The state of the system is defined by the number of white balls in the first box. There are four states: The figure above depicts both boxes the left one being the one in which we count the white balls. Based on the description of the experiment we can declare that this is a discrete time Markov chain. Obviously, if we are ...


4

The first entry in the matrix is $p_{1,1}=1/2$. It means that with $50$% probability you stay in state $1$ for one time step. To the right of $p_{1,1}$ you have $p_{1,2}$ which is the probability that you go from state $1$ to state $2$. Each entry $p_{i,j}$ means that you go from state $i$ to state $j$ where the $i$ is the row and the $j$ is the column. "...


4

To understand the matrix, you need to understand what the states mean. In this case, State 0: The previous flip was not a head. State 1: The previous flip was a head. State 2: You have seen two heads in a row. Assume you are in state 0. You flip a coin. With heads you go to state 1, and with tails you stay in state 0. So we stay in state 0 with ...


4

One simple way: mention $P$ is bidiagonal. Let $\mathbf{P}$ be the $n\times n$ bidiagonal matrix $$\mathbf{P}=\begin{bmatrix} 1-p & p & & & & \\ & 1-p & p & & & \\ & & & \ddots & & \\ & ...


4

Maybe a good starting point would be to look at a simple example. The eigenvalues of the transition matrix $\begin{bmatrix} \frac{1}{2} & 1 \\ \frac{1}{2} & 0\end{bmatrix}$ are: $1$, with eigenvector $\begin{pmatrix} \frac{2}{3} \\ \frac{1}{3} \end{pmatrix}$ (the principal eigenvalue giving the stationary Markov distribution) $-\frac{1}{2}$, with ...


3

We have: $E_0 = 1 + \frac{1}{2}E_0 + \frac{1}{2}E_1$ $E_1 = 1 + \frac{1}{2}E_0 + \frac{1}{2}E_2$ $E_2 = 0$ This simplifies to $E_0 = 6$, which tells you the expected number of flips at the start of the game. We also have $E_1 = 4$ (the expected number of steps when you have already flipped a head). If you want to use the Markov chain instead: $N = (I - ...


3

The row sums of a symmetric matrix are equal to the column sums. For a Markov transition matrix, the row sums are all equal to 1, so for a symmetric Markov transition matrix the column sums are also all equal to 1. But this means that the uniform distribution (a vector all of all ones, rescaled to add up to one) is a stationary distribution. Since the ...


3

Let $T = (t_{ij})$ be an $n \times n$ transition matrix (i.e., $t_{ij} = \mathbb{P}(X_n = j \mid X_{n-1} = i)$). Let $G = (V, E)$ be the graph with vertex set $V = \{1, \ldots, n\}$ and edge set $E \subset V \times V$ such that $(i,j) \in E$ if and only if $t_{ij} \neq 0$. Test if the Markov chain is ergodic Run Tarjan's algorithm on $G$. The Markov chain ...


3

Take a simple system $\dot{x} = x$, then $A=1$ and $\phi(t,t_0) = e^{t-t_0}$. That is $A$ is constant, but the system response is time varying. In general this will be the case unless $A=0$. If you have $\phi(t,t_0)$ in general you have $\dot{\phi}(t,t_0) = A(t) \phi(t,t_0)$, or $A(t) = \phi(t,t_0)^{-1} \dot{\phi}(t,t_0) = \phi(t_0,t) \dot{\phi}(t,t_0)$, so ...


3

$\newcommand{\tr}{\operatorname{tr}}$ The sum of the numbers on the diagonal of a matrix $A$ is called the trace of $A$ which is denoted by $\tr(A)$. The trace of a product of three square matrices of the same size $A,B,C$ satisfies $$\tr(ABC)=\tr(BCA),\qquad \text{(cyclicity)}$$ Then if $A$ and $B$ are similar matrices, i.e. there exists an invertible ...


3

I think we can make a transition matrix with only $5$ states. The $5$ states will be $0, 1, 2, 3, 4$; each corresponds to the number of Jacks we have in our hand. $$ \begin{array}{c c} & \begin{array}{c c c} 0 \ & 1 \ & 2 \ & 3 \ & 4 \ \\ \end{array} \\ \begin{array}{c c c}0\\1\\2\\3\\4 \end{array} & \left[ \begin{array}{c c c} \...


3

Both $\ \big(a_t,b_t,c_t,d_t\big)\ $ and $\ \big(a_t,b_t,c_t,d_t,f_t\big)\ $ are discrete-time, time-homogeneous Markov chains, so you can obtain the distribution of $\ f_t\ $ (at least for moderate values of $\ t\ $) by using the standard procedure for obtaining the state distribution of such chains. In fact, since $\ f_t\ $ is a deterministic function of $\...


3

The eigenvectors corresponding to the non-one eigenvalues simply do not correspond to probability distributions; they have both negative and positive entries. For example, $$\begin{bmatrix}0.1&0.9\\0.9&0.1\end{bmatrix}$$ has an eigenvalue of $-0.8$ corresppnding to the eigenvector $(1,-1)$.


2

Since $f$ is one-to-one, the new sequence $f(X_n)$ is still a Markov chain. To prove this, argue that the transition matrix is the same as before, only the rows and columns are labelled with the new state space $f(0), f(1), f(2)$ instead of $0, 1, 2$.


2

Just write the new basis' elements as linear combinations of the old basis: $$\begin{align*}&1=\color{red}1\cdot1+\color{red}0\cdot x+\color{red}0\cdot x^2+\color{red}0\cdot x^3\\{}\\ &2-x^2=\color{red}2\cdot1+\color{red}0\cdot x+\color{red}{(-1)}x^2+\color{red}0\cdot x^3\\{}\\ &x+x^3=\color{red}0\cdot 1+\color{red}1\cdot x+\color{red}0\cdot x^2+...


2

The easier way to approach this problem is to take advantage of linearity of expectation by expressing the transit time as a sum of i.i.d. geometrically-distributed random variables, as in Daniel Xiang’s answer, but it can certainly be attacked the way you propose: by computing the probability that it takes a given number of time steps to reach the end and ...


2

Let $T_i$ be the time it takes to reach node $i$ for $i = 1, ..., 5$. Then the total time $T$ it takes for one to reach node 6 from node 0 can be written as a sum \begin{align*} T = T_1 + \dots + T_5 \end{align*} So the expected value of $T$ is \begin{align*} ET = \sum_1^5 ET_i \stackrel{(*)}{=} \sum_1^5 \frac{1}{p} = \frac{5}{p} \end{align*} $(*)$ Since ...


2

You are given the transition matrix for what happens on exiting permanence states.   That is not for what happens at the end of any minute; only what happens when a state change does finally occur. You are also informed that the permanence time (a count of minutes) is itself a geometric random variable with given means.   That tells you the ...


2

"Firstly I wanted to check if they have a mistake in the solution. So for the transition matrix, the element p11 should be 0 and p12 should be 1, but they have it the other way round so I just wanted to check whether this was a mistake or not." You are correct. $p_{1,1}$ should be $0$ and $p_{1,2}$ should be $1$. "Secondly If a set is a communication class ...


2

Define $F_n$ to be the indicator random variable which has value 1 if at $n^{th}$ step white ball Chosen from the first ball and else 0. Similarly Define Indicator random variable $S_n$ for second urn Now,To check Markov property we need to check P($X_n$= j|($X_{n-1}$,$X_{n-2}$,..$X_0$)=($i_{n-1}$,$i_{n-2}$,...$i_0$))=P($X_n$= j|$X_{n-1}$=$i_{n-1}$) ...


2

Following the hints by @Ian, we present an answer, using the ergodic theroem and and the Monte Carlo Simulation. First of all, we easily see that there is a unique invariant distribution $\pi:$ $$\pi=\pi P\iff \pi(k)=\frac{1}{2}~\pi(k-1),~k\in \mathbb{N} \Rightarrow~ \pi(k)=\frac{1}{2^k}\pi(0),~k\in \mathbb{N}$$ and $\displaystyle \sum_{k=0}^{\infty}\pi(k)=...


2

Hint: If $U$ is the transition matrix from the standard basis to the basis $S$ ( i.e. the matrix that has as columns the vectors $u_i$), and $V$ is the transition matrix from the standard basis to the basis $T$, than the transition from $S$ to $T$ is given by the matrix $V^{-1}U$.


2

I think it is easy to see that having all the $\pi_i$ equal to each other gives a solution. I assume you also want $\sum \pi_i$ to be $1$. This would give the solution $\pi_i = \frac{1}{23}$. I didn't explain why that is the unique solution.


2

There is the intuition, and then there's the formal description. Here is the formal description: Note that for any matrix $M$, the product $[1,1,\ldots, 1]\cdot M$ computes a row vector containing the column sums of $M$. For any matrix of transition probabilities $T$, a stationary distribution is a row vector of probabilities $\pi$ such that $\pi \cdot T =...


2

You want $\mathbf P_{00}(\mathbf P_{00}\ p_0+\mathbf P_{10}\ p_1+\mathbf P_{20}\ p_2)$, where $p_0,p_1,p_2$ are the probabilities for the initial state of the system being $A,I,O$.


2

Yes, it can be done reasonably simply. All you have to do is to carefully consider the true transition probabilities. Consider, for instance, the probability of going from state 1 to state 2 in a single step. If the "original" matrix is used, then this probability is $0.3$. If the switch happens, then this probability is $0.5$. Regard this as a holistic ...


2

This is not a complete answer, but it's too long for a comment. One way to do this, and I don't know that it's the best way, is to use the fact that $e^{At+Bt}=e^{At}e^{Bt}.$ Now write $Q=D+N$ where $Q$ is a diagonal matrix and $N$ is nilpotent. That is, $D$ is the matrix with the given diagonal elements, and $N$ has all the off-diagonal elements. Since $...


2

A denotes active(running) and R denotes Repaired. The transition matrix is given by the states of these two machines to coexist in the system. $$ \left(\begin{array}{cc} &A_1A_2 & A_1R_2 & R_1A_2& R_1R_2\\ A_1A_2 & (1-p)^2 & (1-p)p& p(1-p)& p^2\\ A_1R_2 &0 & (1-p)& 0& p\\ R_1A_2 &0 & 0& (1-p)&...


Only top voted, non community-wiki answers of a minimum length are eligible