# Tagged Questions

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

30 views

### Why do we have these probability functions for this Markov Chain?

The following shows one of the questions we were given in lectures a while back: We have been given the following solutions to this question: I'm rather confused by these. Take, for example, the ...
28 views

### What is an example of a second-order markov chain? [closed]

I'd like to see an example of a second-order markov chain. Haven't found one over google or in any of my textbooks
29 views

### Sufficient condition for a measure to be invariant

Given a Polish metric space $H$ and a Borel probability measure $\pi$. Let $\mathcal B_b(H)$ be the set of bounded measurable functions on $H$, and $L^2(H, \pi)$ be the set of square integrable real-...
38 views

77 views

### Distribution of a process dependent on a Markov chain's states

Consider a Markov chain $X_t$ with state space $\{0,1\}$, initial distribution $$\begin{array}{l} \mathbf{P}(X_0=1)=\lambda \\ \mathbf{P}(X_0=0)=1-\lambda \end{array}$$ and transition ...
39 views

23 views

### Kolmogorov forward and Backwards equation interpretation

Let $\lambda_i$ be the sojourn rate of state i, $q_{ij}$ be the transition rate form i to j, and $p_{ij}$ be the transition probability from i to j. The Kolmogorov Forward and backwards equation are ...
42 views

16 views

### Does an embedded discrete-time Markov chain preserve its properties in continuous time?

Given a discrete-time Markov chain without independent increments, is the embedding of it into a continuous time Markov chain (i.e. via the use of exponential waiting times) an example of a continuous ...
65 views

### Example of a Continuous-Time Markov Process which does NOT have Independent Increments

1. Given a discrete-time Markov chain without independent increments, is the embedding of it into a continuous time Markov chain (i.e. via the use of exponential waiting times) an example of a ...
34 views

### Markov chain - Stationnary distribution - Unique

Consider the following respective infinitesimal generators of Markov chains in continuous time: A=\begin{bmatrix} -4 & 1 & 3 \\ 3 & -5 & 2 \\\ 0 & 3 & -...
26 views

### Show that a matrix satisfying certain conditions is non-singular

I have a square matrix $A$ satisfying the following conditions: The elements on the diagonal are negative; All other elements are non-negative; All row sums are less than or equal to $0$; There is ...
7 views

### What is the probability of this Markov Jump process remaining in this state?

Suppose you had a time homogeneous Markov jump processed defined by the following transition diagram I'm assuming that this means that the process remains in state $0$ for time $t$ with probability ...
25 views

### Period of an irreducible Markov Chain is given by the number of eigenvalues with unit modulus

Suppose $\{X_n\}$ is an irreducible Markov Chain on finite state space $S$. Then, the number of eigenvalues of the transition matrix with unit modulus is precisely equal to the period of the chain. ...
11 views

### What does the notation $P_{\overline{MM}}(t)$ mean in this context?

The notation $P_{\overline{MM}}(t)$ is used in part (iii) of the following question: I'm unsure of exactly what this notation represents. My guess would be that it represents the probability that a ...
13 views

### Pure jump process

I'm having touble understand the pat of the solution that I have underlined in green for b)
32 views

### Markov Process graphical representation

I don't understand how the picture has been constructed. Specifically how $\mu^{11}=-(\mu^{12}+\mu^{13}+\mu^{14})$ and $\mu^{44}=-\mu^{43}$ has been graphically represented. Here $\mu^{ij}$ is the ...
23 views

32 views

### Constructing transition graph from transition matrix

Ok so for this question I'm having trouble understanding how the transition graph has been drawn from the given transition matrix. This is what I understand and hopefully someone can correct the flaws ...
39 views

### Soft Question - book recommendation - Stochastic Processes

My mother language book on stochastic processes is pretty much complete(~500 pages) but would like another one in English, to have in my library. I'm looking for a similar book containing the ...
30 views

### Is it correct to say the expected waiting time is infinite?

Say I have some process that starts in a state in the set of states $S$, each possible starting state having non-zero probability. Some (most) of the starting states eventually result in reaching a ...
65 views

### Prove that $Y_n=X_{n-1}X_n$ is a markov chain

Let $\{X_n\}_{n=0}^\infty$ a sequence of discrete random variables independent identically distributed. Let $Y_n$ such that $Y_n=X_{n-1}X_n$ for all $n\ge 1$ Is $\{Y_n\}_{n=0}^\infty$ a markov chain?...
23 views

### What is the (practical) difference between a stationary distribution and an equilibrium distribution of a MC?

I know that, for a Markov Chain, a stationary distribution is the (row) vector $\pi$ such that $\pi \cdot P = \pi$, where $P$ is the one-step transition matrix for the MC. Intuitively, I assume that ...
68 views

### prove homogeneous markov chain

$Y_0, Y_1,Y_2,\dots$ are independent and identically distributed random non-negative integer outcomes. Let X_0 = Y_0 $Let$X_0 = Y_0$and$X_n = X_{n-1} - Y_n$if$X_{n-1}>0$, else$X_n = X_{n-1}...
23 views

### How to interpret 'arbitrary customer'

My question is about how to interpret 'arbitrary customer' in the following scenario (see question 2. listed below): "At a single server service station two types of arrivals occur. According to a ...
There's in a banc two identical queues and totally separated : these are two queues of type $M/M/1$. For each of them, the arrivals are separated by exponential times of parameter $\nu$, the time ...