# Tagged Questions

7 views

### Computing the PMF for N(t) in a renewal process

I'm in a stochastic processes course, and we just started on renewal theory. Unfortunately, we skipped the section on queuing theory, and nearly example in my textbook for renewal processes uses ...
7 views

### First property of discrete time homogenous markov chain

I'm trying to understand the properties of a DTHMC. I am having trouble understanding with the first one. My textbook says - "$X_t$ takes values in $X$ for all $t$ (i.e. $X_t$ is a random variable ...
13 views

107 views

### understanding submartingale proof with discrete state space

I am reading a text about branching markov chains: My question is about the first half of page 8 where $Q(t)$ is proven to be a submartingale. Briefly the used notation: $t$ is discrete time, $n(t)$ ...
31 views

### markov spectral radius independent of states?

Let $\Pi$ be a stochastic matrix of an irreducible markov chain. We define the spectral radius of $\Pi$ as: $\rho(\Pi) := \limsup_{n \to \infty} \left( \pi^{(n)}_{(a,b)} \right)^{\frac{1}{n}}$ Why ...
46 views

### Proving a chain is aperiodic, and finding a stationary distribution.

We have an irreducible Markov chain with a not necessarily finite state space. It has a transition matrix $P$ such that $P^2=P$. Prove (1) the chain is aperiodic, and (2) prove $p_{ij}=p_{jj}$ ...
17 views

### Question involving an invariant measure on a Markov chain

Suppose $\mu$ is an invariant measure for a Markov chain with state space $S$ with $\mu(i)p_{ij}=\mu(j)p_{ji}$ $\forall i,j \in S$. Describe a Markov chain with this property. Also, show that $\mu$ is ...
19 views

### Transition rate matrix from transition probability matrix

If I have a $2 \times 2$ continuous time Markov chain transition probability matrix (generated from a financial time series data), is it possible to get the transition rate matrix from this and if ...
75 views

26 views

### Markov renewal process vs Markov Jump process

The venerable wikipedia for "Markov renewal process" says that: "a Markov renewal process is a random process that generalizes the notion of Markov jump processes". So what's the definition of a ...
56 views

### When are stable continuous time Markov chains Feller? Always?

This is a question is similar to this 2 year-old one that never got answered (truthfully it's pretty much the same question except that I'm adding a bit more detail and the assumption that the $Q$ ...
58 views

Let $dX_{t} = m_1(l_1-X_{t})dt+\sigma_1 dW_{t}$ and $dY_{t} = m_2(l_2-Y_{t})dt+\sigma_2(\rho dW_{t}+\sqrt{1-\rho^2}dW_{t}^{1})$ where the $m_i$'s, $l_i$'s and $\sigma_i$'s are constants, $\rho \in ... 1answer 87 views ### Elementary proof of geometric / negative binomial distribution in birth-death processes The birth-death process concerns a population of$n_0$individuals, each of which reproduce and die at a constant rate as time$t$increases from$t=0$. Each individual splits into two individuals ... 0answers 18 views ### Finite state Markov chains and expectations Let$X_t$be a finite state Markov chain with generator matrix$Q$. For a give function$f(x)>0$define: $$u(t, i) = \mathbb{E}\left[\int_t^T f(X_s) \mathrm{d} s| X_t = i\right]$$ Are there any ... 2answers 38 views ### Counterintuitive Markov chain problem In class, my professor said that given a Markov chain$\{X_k\}$it intuitively should be true that$P(X_{k+1} = x_{k+1} \, \mid \, X_0 = a_0, \dots, X_{k-1}= a_{k-1}) = P(X_{k+1} = x_{k+1}\, \mid ...
51 views

We know that if $\{X_n\}$ is a Markov chain, then $X_{n+1}$ is independent with the past states $X_0,\ldots,X_{n-1}$ given current state $X_n$, that is ...