# Tag Info

3

If you absolutely don't want to diagonalize, you could do something like this: First convince yourself that the sequence must converge. For example, instead of having a sequence, imagine just having three numbers and successively replacing one of them with the average of all. If you keep the details straight, you can then see that the maximum of the numbers ...

2

The distribution for the number of time steps to move between marked states in a discrete time Markov chain is the discrete phase-type distribution. You made a mistake in reorganising the row and column vectors and your transient matrix should be $$\mathbf{Q}= \begin{bmatrix} \frac{2}{3} & \frac{1}{3} & 0 \\ \frac{2}{3} & 0 & ... 2 It appears that the transition semigroup need not be Feller. An example: Denote the non-negative integers by \mathbb{N}. Define, for i,j\in\mathbb{N},$$q(i, j) =\begin{cases} 0 &\mbox{ if }i =0\\ i^2(\delta_{i-1, j}-\delta_{i,j}) &\mbox{ otherwise.}\end{cases}$$Then the backward equations are$$p_t'(0,0) = 0$$and$$p_t'(i, i-1) = ...

2

As you noted, away from zero the bias is uniformly negative. This suffices to guarantee recurrence, that is, that the process hits zero with full probability. In a nutshell, choose $N\gt2(k+1)c$, and replace the dynamics on $b\geqslant N$ by $b\to b+k$ with probability $c/N$ and $b\to b-1$ with probability $1-c/N$. Starting from any $b(0)\geqslant N$, this ...

2

It is a theorem of Wielandt that the number you are looking for is $(r-1)^2+1$. A way to achieve this is to have $p_{12},p_{23},p_{34},\dots,p_{k-1,k},p_{k,1},p_{k,2}$ positive and all other $p_{ij}$ zero. A reference is H Wielandt, Unzerlegbare, nicht negative Matrizen, Math Z. 52 (1950) 642-648. Perhaps a better reference is Hans Schneider, Wielandt’s ...

2

You have to solve the following recursive relations. Let $h(k)$ be the expected number of steps until your reach the (absorbing) state $4$ when you are in state $k$, for $k=1,2,3,4$. So we have that $$h(4)=0$$ because when you are already in $4$ you need zero steps to reach $4$. Then for $k=3$ $$h(3)=1+0.75h(3)+0.25h(4)$$ because when you are in state 3 you ...

1

We show how to find an answer using not much machinery. Matrices, or recurrences, would make it easier. At any time, we can be in one of two states, $A$ or $B$, where State $A$ means we are at vertex $1$, and State $B$ means we are at one of the $3$ other vertices. At time $0$ we are in State $A$. If at time $n$ we are in State $A$, then with probability ...

1

For example, the entry in row "1", column "3" is the probability of going from "1" to "3" which is 0.4. We can't decrease $X_n$ so some values are 0. Each row must add up to 1. $$\begin{matrix} & 0 & 1 & 2 & 3\\ 0 & ? & ? & ? & 0.4\\ 1 & 0 & ? & ? & 0.4\\ 2 & 0 & ... 1 A little bit more details. What is the probability that in a time interval (0,t) exactly one person will have a success. Well first pick the person (say P1) that it will be call their child P2 and everyone else E, and condition on exactly when the success happens. Call that pdf of the first birth time of P1 p_1(s).$$P(\text{P1 has 1 birth & P2 and ...

1

Of course. But the $C_i$'s you get can be split again in stable subsets. The point here is that for every $i$, $P^d$ restrained to $C_i$ is aperiodic when $d$ is the period of the chain. In other words, $C_i$ can't be split in several subsets stable under a power of $P$, giving nice properties.

1

The idea is that $X_k$, whatever it is, has to be something, so you sum over it: $$P(X_{k+1} = x_{k+1} \mid X_0 = a_0, \dots, X_{k-1} = a_{k-1})$$ $$= \sum_{x_k} P(X_{k+1} = x_{k+1}, X_k=x_k \mid X_0 = a_0, \dots, X_{k-1} = a_{k-1} )$$ $$= \sum_{x_k} P(X_{k+1} = x_{k+1}\mid X_k=x_k, X_0 = a_0, \dots, X_{k-1} = a_{k-1} )\cdot P( X_k=x_k \mid X_0 = a_0, ... 1 For a counterexample consider the Markov chain \{X_n\} described here by Did. Assume everything to be the same as in the example, but instead of states 0, 1 and 2 consider the states -1, 1 and 2. Now the function g(x)=x^2 lumps together states -1 and 1. That is, we have that g(-1)=g(1)=1 and g(2)=4, exactly as described in the ... 1 Firstly separate the states in communication classes. There are two communication classes which are determined as follows. Start from state 0. Which states can you visit and then return to state 0? You can make following transitions$$0\xrightarrow{0.5} 2 \xrightarrow{0.9} 0$$So 0 and 2 communicate, which means that they belong to the same ... 1 There are some common (and very useful) results concerning recurrence and transitivity of states which you might know and/or could use: All states in an irreducible set are either all recurrent or all transient. State i is transient if and only if $$\sum_{n=1}^{\infty}p_{ii}^{(n)} < \infty$$ State i is recurrent if and ... 1 There is a simple algorithm: let P the transition matrix. let A=I + P, j = 1 replace each non 0 entry by 1 replace A by A^2, j by 2j Go back to 2. until j\ge n Then the graph is connected iff every entry of A is 1. At each iteration, at the end of step 4. the non zero entries of A are the state you can go to with \le j ... 1 To show that M is not Markov, one can consider two different paths of M between the times 0 and 4: If (M_n)_{0\leqslant n\leqslant4}=(0,1,1,1,2), then S_2=0 hence S_3 is conditionally uniformly distributed on \{-1,1\} and the last step 1\to2 has conditional probability \frac12\cdot P(X_4=1)=\frac14. If (M_n)_{0\leqslant ... 1 The grinder can be in two states -- either in use, or not in use. The period of being in use lasts for time 1/\mu per minute on average, so the rate here is \mu. When the grinder is not in use the operator arrives at rate 0.5 per minute (1 per 2 minutes). Therefore we have a continuous time Markov chain on the state space {in use, not in use} with ... 1 It this case, if for example m_1\neq m_2, \sigma_1\sigma_2\rho\neq 0, no, because the drift is not a function of X_t + Y_t. In a general situation, when X,Y are two non independant Markov chains, X+Y is not. To understand this with a more simple example, in a discret time setting, take for example (X_n) be a simple random walk on \mathbb Z, and ... 1 Not always. For a slightly degenerate example, assume that X, Y_1 and Y_2 take values in \{+,-\} with Y_1=Y_2=+ if X=+ and Y_1=+ and Y_2=- if X=-. The paths of the triple (X,Y_1,Y_2) are (+,+,+) and (-,+,-) hence the distribution of Y_2 conditionally on (X,Y_1) depends on X, that is, (X,Y_1,Y_2) cannot be a Markov chain. For ... 1 Usually, my approach to this kind of question is to solve a very simple recurrence. Just look to  E_{0} N(3) but conditioned in each of the two possible first steps to get (I will write N instead of N(3)$$E_0 N = \frac{2}{3}E_0N + \frac{1}{3}E_1N +1 The term +1 shows up because once you had walked one step you have to add this step to the ...

1

To give the answer you quote, you need a stronger version of (3) so that the states at each time point are independent of each other. Hints: What is the probability that at any particular time point the object has a partiuclar state $x$? What is the probability that at any particular time point the object does not have state $x$? What is the probability ...

1

You give four equations for three unknowns. Fortunately, there is a point of major interest : substracting the first equation from the third equation gives the second equation. Then, either the first or the third equation must be discarded. So we have now three linear equations for three unknowns. Suppose that we discard the first equation. Extract \$x = 5 ...

Only top voted, non community-wiki answers of a minimum length are eligible