# Tag Info

3

First part: For every $n$, the simple Markov property at time $nK$ yields $$P_j(T_i\gt(n+1)K\mid T_i\gt nK,X_{nK}=k)=u(k),\qquad u(k)=P_k(T_i\gt K),$$ and $u(\ )\leqslant1-\varepsilon$ uniformly by hypothesis hence $$P_j(T_i\gt(n+1)K,X_{nK}=k)\leqslant(1-\varepsilon)P_j(T_i\gt nK,X_{nK}=k).$$ Summing these over $k$ yields ...

3

There can be no oscillations, because then there would be a state that you visit infinitely many times. Each time you visit that state, there is a positive probability for the process to go extinct. For example, just the probability of having no children in the next generation. Denote this probability $p > 0$. Since you visit this state infinitely many ...

2

Since the Markov chain is irreducible, it is possible to get from state $i$ to state $j$, so $p_{ij}^{(k)} > 0$ for some $k$. Then $p_{ij}^{(n+k)} \ge p_{ii}^{(n)} p_{ij}^{(k)}$, so $$\sum_{n\ge 0} p_{ij}^{(n)} \ge \sum_{n\ge k} p_{ij}^{(n)} = \sum_{n\ge 0} p_{ij}^{(n+k)}\ge p_{ij}^{(k)} \sum_{n \ge 0} p_{ii}^{(n)} = \infty$$

2

More generally. Let $(X_n)_{n\geqslant0}$ be defined by $X_{n+1}=G(X_n,Z_{n+1})$ for every $n\geqslant0$, where $(Z_n)_{n\geqslant1}$ is i.i.d. and independent of $X_0$. Then $(X_n)_{n\geqslant0}$ is a Markov chain. If need be, one can write down the transition probability of $(X_n)_{n\geqslant0}$ as $$P(X_{n+1}\in A\mid (X_k)_{0\leqslant k\leqslant ... 1 Given some thought, I've realized the answer is simple. Let W = x + y + z. Because this is a Markov chain, for all M^nv, x_{M^nv}+y_{M^nv}+z_{M^nv} = W. The equation this line fits is x + y + z = W. Thanks for the help everyone. I'm amazed I did not see this last night. 1 The classical conjugate gradient method (CG) for linear algebraic systems is equivalent to applying the Lanczos algorithm on the given matrix with the starting vector given by the (normalized) residual of the initial approximation. The coordinates of the approximate solution in the orthonormal basis generated by Lanczos are obtained by solving certain linear ... 1 Here is a complete solution to (d), which is provided in the hope that the OP will emulate it for the other items. Consider the results (Z_n)_{n\geqslant1} of the die, these are some i.i.d. sequence and, for every n and k,$$[B_n=k]=[Z_{n+k}=6]\cap\bigcap_{i=1}^{k-1}[Z_{n+i}\ne6].$$The dynamics of (B_n) is as follows: If B_n=k with ... 1 You know that \lim\limits_{n \to \infty} P(X_n=i) = \pi(i) and that \sup_n P(X_n = i)P_{ij} = \sup_n |P(X_n = i) P_{ij}| \leq P_{ij}. This means that you use the dominating function i \mapsto P_{ij} noting that \sum_i P_{ij} = 1 (hence i \mapsto P_{ij} is in L^1) and apply the DCT. Does this clarify the use of the DCT? 1 The simplest approach: X_0=Y_0 and, for every n,$$X_{n+1}=A(X_n,Y_{n+1}),\qquad A(x,y)=\max\{x,y\}.$$And now, watch the results fall in line like dominoes: Initial distribution: the distribution of Y_0 Markov property: obvious since X_{n+1} is a deterministic function of the present state X_n and of a new input Y_{n+1} which is independent ... 1 Here's a rough estimate which may give an idea of how to get a proper answer. Let X_i(t) be the number of colors such that there are i balls of that color. Let the level at time t, denoted by L(t), be the largest i such that X_i(t)>0. Suppose L(t)=i. We want the probability that L(t+1)=i+1. At time t, there are i X_i(t) balls of ... 1 Two classes of models: Markov chains of higher order, and Varying Length Markov Chains (VLMCs, also known as Variable-Order Markov Models). 1 All of your statements are true. For the first, you can use a purely linear algebraic fact, that a stochastic matrix always has an eigenvalue of 1 with a left eigenvector whose entries are nonnegative. For the second, let's say the chain has m communicating classes C_j. If the initial condition is entirely in a single communicating class C_j (i.e. ... 1 Direct computations show that, for every n\geqslant1, E(N_{t+1}-N_t\mid N_t=n)=\sum\limits_{k=0}^\infty ka_k-R\sum\limits_{k=0}^\infty a_k (assuming the series \sum\limits_kka_k converges absolutely, say). This drift does not depend on n\geqslant1. The formula for E(N_{t+1}-N_t\mid N_t=0) is different since the transitions from state 0 are ... 1 As explained in the comments, differentiating exponentials of matrices such as \bar P(s,t) is slightly more complicated than in the scalar case since, quite generally,$$\left.\frac{\mathrm d}{\mathrm dt}\mathrm e^{A+tB}\right|_{t=0}=\sum_{i\geqslant0}\sum_{k\geqslant0}\frac{A^iBA^k}{(i+k+1)!}, which is neither $B\mathrm e^A$ not $\mathrm e^AB$ in ...

Only top voted, non community-wiki answers of a minimum length are eligible