# Tagged Questions

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

94 views

19 views

### Understanding stochastic matrices

We start a game with 2 euros, i.e. at time 0 we have 2 euros. At time $t=1,2,...$ we play a game with a stake of 1 euro and with odds of winning $p$ (hence odds of losing $1-p$). We define $X_t$ at ...
31 views

### Stochastic calculus for continuous time Markov chains

I have absolved a course on stochastic analysis, i.e. integrals with respect to the brownian motion. Now I know that there is a theory of stochastic calculus for diskrete matringales, however I was ...
22 views

26 views

### Stochastic process $X(t)=W_tX(t-1)$ with $\left\{W_t\right\}_{t=1}^n$ iid row stochastic matrices

I have been struggling for a while with the following problem. Consider a sequence of iid row stochastic matrices $\left\{W_t\right\}_{t=1}^n$ and the linear dynamical system $X(t)=W_tX(t-1)$ with ...
55 views

45 views

### Markov chain with absorbing states?

Let's say I have $5$ states (state $2$ to $6$, state $1$ is missing) when time$=0$, and $6$ states (state $1$ to $6$) when time$=1$, and now I want to calculate the transition matrix. Does it mean ...
40 views

61 views

### Markov Chain Application - flipping coin heads, tails, finished

Suppose that we are flipping coins iteratively, until we get tails two in a row. Define three states: Heads, Tails, and Finished. Suppose that the probability of getting a head is $p$, and ...
25 views

### When is a markov process not conservative where do we need $P(t,x;I)$ to be continuous to the right?

We find on Petr Mandl's book Analytical treatment of one-dimensional Markov processes page 10 the following definition of conservative markov process My question concerns the proof of theorem 4 ...
35 views

52 views

### If $N_t$ is a Poisson process and $Y\in\{-1,1\}$, then $X_t = Y(-1)^{N_t}$ is a Markov process

Let $Y$ be a random variable with values in $\{-1,1\}$ independent of a Poisson process $\{N_t\}$ with intensity $\lambda>0$. Set the process $X = \{X_t\}$ by $X_t=Y(-1)^{N_t}$. Show that $X$ is a ...
71 views

### markov chains and coin flips

A coin that comes up heads with probability p is continually flipped until the pattern T T T H appears. Let X denote the number of flips, find EX. If I use Markov chains is there a simpler way to ...
21 views

### Why do we look only for the first and second derivative when dealing with diffusions?

I would like to understand, from an analytical point of view, why is it that we only take the first and second derivatives into account when computing the generator of a diffusion. This question is ...
38 views

### Importance of uniform stationary distribution

When I study Markov chain (or sampling) related papers, most of them emphasize "uniform stationary distribution". But, I can't sure why it is important for Markov chain problems or randomized ...
28 views

### Transition Probability of M/M/1 Queue given any constant observing period T

I am trying to find the transition probability for $M$/$M$/$1$ queue given any constant observing period $T$ (if $T$ go to infinity the transition matrix will degenerate to a matrix with identical ...
35 views

### The differences between the return times to a recurrent state of a discrete Markov chain are independent and identically distributed

Let $(\Omega,\mathcal A)$ be a measurable space and $\mathbb F=(\mathcal F_n)_{n\in\mathbb N_0}$ be a filtration on $(\Omega,\mathcal A)$ $E$ be an at most countable set equipped with the discrete ...
39 views

### Markov process, compute how much time spent in each state in average before absorption.

This problem has three states for a person, which are either employed, unemployed or early retirement. The probability that a working person goes unemployed is 0.2 (ie with intensity 0.2 per year). ...
23 views

### Fluctuation of a martingale conditioned to return

Consider a martingale $M_t$ on $\mathbb{Z}$ starting from $M_0=0$ and such that $Var[M_t] \leq C \, t$, where $C>0$ is some constant. For a given $n \in \mathbb{N}$ and $t \leq n$, define a process ...
19 views

### Property of an irreducible Markov Chain

How can we prove that if a Markov Chain is irreducible (does not contain any closed set), then every state can be reached from every other state in the chain ?
Suppose to have the following situation: At a bar at each time unit arrives a certain number of customer with probabilities $p_1,p_2,...,p_n$. In the bar there are 3 bartenders so 3 customer can be ...
Let $E$ be an at most countable set equipped with the discrete topology $\mathcal E$ $X=(X_n)_{n\in\mathbb N_0}$ be a discrete Markov chain with values in $(E,\mathcal E)$, distributions \$(\...