# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

15 views

### A Markov Chain Flea Problem

A Flea moves around the vertices of a triangle in the following manner: Whenever it is at vertex i it moves to its clockwise neighbor vertex with probability pi and to the counterclockwise neighbor ...
10 views

### variance of number of steps in markov chain (rook move to top right)

I encountered this problem while studying Markov chains and I want to calculate the variance of the problem, i.e. variance of number of steps that a random walker rook make to reach from down-left ...
33 views

### Gambler's ruin and Markov Chain, coin toss and stakes

I'm considering a classical problem about Markov Chains: A gambler has $£8$ and wishes to get to $£10$. A coin is tossed repeatedly : if it comes down tails, the gambler loses his stake, and if it ...
22 views

### A die whose score cannot be as before (Markov chains)

A die is "fixed" so that each time it is rolled the score cannot be the same as the preceding score, all other scores having probability $1/5$. Given that the first score is 6, what is the probability ...
20 views

### Markov Chain: Moving on a circle

A particle moves on 12 points situated on a circle. At each step it is equally likely to move one step in the clockwise or in the counterclockwise direction. Find the mean number of steps for ...
8 views

### Expected number of visits to a state of a Markov chain up to a certain time

Let $P=\{p_{ij}\}$ be a stochastic matrix (with rows and columns indexed by a countable set) and let $p^{(k)}_{ij}$ be the entries of $P^k$. I'm trying to prove that, if the associated Markov chain is ...
22 views

### show is markov chain [on hold]

suppose: that : X=({X}{n}){n\geq 0}: is: M.C(\lambda ,P): y : f:IxI\rightarrow I a function. denote by ${f}^{-1}(j):={i\in I:f(i)=j}\: \: y \:$ suppose for all $i,j \in I$ such that $f(i)=f(j)$ ...
23 views

I need some help verifying that my understanding of steady state distribution is indeed correct. I have a transition diagram (model). With around 100 states and 6 variables. I have used a software ...
22 views

### Using long term Markov Chains in Excel to compute time spent in state

I'm trying to calculate the uptime of a computer state in Excel. I'm in way over my head, and it took me over a day to identify my problem as a Markov Chain. The question is: "X pellets per second ...
14 views

### Markov Chains (State transitions)

I was wondering which part I am misunderstanding about the individual-by-individual updating scheme from the book of Jackson M. (Social and Economic Networks, 2008) . The full transition matrix in the ...
13 views

### Finding the infinitesimal generator of a M/M/2 queue [on hold]

I have a M/M/2 queue with a total population of 5. The arrival times are independent exponential random variables with mean of $\lambda$ and the service times have a mean of $\mu$. The initial number ...
19 views

### Setting up and Solving Kolmogorov Forward Equations

Consider a Markov Chain with $3\times 3$ generator matrix: $$G = \begin{bmatrix} -1 & 1/2 & 1/2 \\ 1/2 & -1 & 1/2 \\ 1/2 & 1/2 & -1 \end{bmatrix}$$ What are the ...
29 views

31 views

### Are $T_4$ and $T_5$ stopping times?

Let $(\xi_n)_{n\in\mathbb{N}_0}$ be a sequence of independent identically distributed random variables that take values in $\left\{-1,1\right\}$ with equal probabilities. Define ...
40 views

### Is this a stopping time or not?

Let $(\xi_n)_{n\in\mathbb{N}_0}$ be a sequence of independent identically distributed random variables that take values in $\left\{-1,1\right\}$ with equal probabilities. Define ...
18 views

### Given initial state and steady state, how do I find the transition matrix?

The initial vector is $x_0 = \{.5, .5\}$ and the steady state is $x_\infty = \{1/9, 8/9\}$. How do I get the transition matrix, such that $$\lim_{p\to\infty} x_0 A^p = x_\infty$$ where $A$ is a ...
37 views

### Show that every finite closed class is positive recurrent

Let $C$ be a finite closed class. Prove or disprove that $C$ is positive recurrent. Note 1: In our lecture we proved that every finite closed class is recurrent. Note 2: (Positive) recurrence is ...
8 views

### Recurrence time of persistent state in a markov chain

Let a Markov chain contains a states and let $E_j$ be persistent. There exists a number $q < 1$ such that for $n \ge a$ the probability of the recurrence time of $E_j$ exceeding $n$ is smaller ...
21 views

### Is it possible to compute these probabilities concerning a 6-digit password using theory of Markov chains? [duplicate]

Consider the situation of decoding a 6-digit password that consists of the symbols A to Z and 0 to 9, where all possible combinations are tried randomly and uniformly. Consider the ...