Tagged Questions
1
vote
1answer
21 views
Mean number of particle present in the system: birth-death process, $E(X_t|X_0=i)$, $b_i=\frac{b}{i+1}$, $d_i=d$
Let $\{X_t\}$ be a birth–and–death process with birth rate
$$
b_i = \frac{b}{i+1},
$$
when $i$ particle are in the system, and a constant death rate
$$
d_i=d.
$$
Find the expected number of particle ...
1
vote
0answers
19 views
Estimate on Galton-Watson process distribution
Let $(Zn)_{n\in \mathbb N_0}$ be a Galton-Watson process, i.e.
$$ Z_{n+1} = \sum_{k=1}^{Z_n}\xi_{n,k},\qquad (\xi_{n,k})_{n\in \mathbb N_0,k \in \mathbb N} \quad \text{i.i.d } \mathbb N_0 \text{ ...
0
votes
1answer
38 views
Calculating probabilities in genetic sequences
I am working with certain recurring sequences in genetics and try to calculate certain probabilities:
Let for instance
$$\langle g_i\rangle :=\{1,1,1,6,1,1,1,6,...,1,1,1,6\}$$ and
$$\langle ...
0
votes
1answer
39 views
Probability of explosion in a Markov chain
I have the following problem: In a chain reaction a particle of a certain kind has probability 4/7 of hitting
a nucleus. If that happens then the particle disappears but 3 new particles of the same
...
3
votes
2answers
60 views
What's the probability that A wins finally
Suppose A has \$2 and B has \$3. They play a game, each game gives the winner \$1 from other. A has a probability $\frac{3}{5}$ to win each game. They play this game until one of them is bankrupt. ...
2
votes
2answers
75 views
$(X_n)$ an irreducible transient Markov chain. Is $f(x) = \mathbb{P}(X_n = x_0 \text{ for some } n > 0 | X_0=x)$ constant?
Let $(X_n)_{n=0}^{\infty}$ be an irreducible transient Markov chain with countably infinite state space $E$. Let $T_x = \inf\{n > 0 : X_n = x\}$. Let $\mathbb{P}_x$ be probability conditioned on ...
2
votes
0answers
121 views
Boundedness of expected reward Markov chain (may be related to discret $M/M/\infty$ queue)
[EDIT]:
I read a bit on $M/M/\infty$ queue and it may not be the right comparison and my notation may be confusing (I'm in discrete time and $\lambda,\mu$ look likes rates when they are probability). ...
0
votes
3answers
235 views
Finding the transition probability matrix, two switches either on or off..
Each of two switches is either on or off during a day. On day n, each switch will independently be on with probability
[1+number of on switches during day n-1]/4
For instance, if both switches are on ...
6
votes
1answer
163 views
What happens to a random walk when we increase the probabilities of going right?
Consider a random walk on the integers where the probability of transitioning from $n$ to $n+1$ is $p_n$ (and of course, the probability of transitioning from $n$ to $n-1$ is $1-p_n$); we assume all ...
0
votes
1answer
20 views
Near-independence of Markov chain states
Let $X(0), X(1), X(2), \ldots$ be an aperiodic irreducible Markov chain on a finite set. My intuition says that if $m$ is a very large number, then $$X(m), X(2m), X(3m), \ldots$$
should be nearly ...
0
votes
1answer
57 views
Mean Duration of Stochastic/Markov Game
An urn contains five red and three green balls. The balls are chosen at random, one by one, from the urn. If a red ball is chosen, it is removed. Any green ball that is chosen is returned to the urn. ...
0
votes
1answer
102 views
Markov Chains Probability
A Markov chain $X_0$, $X_1$, $X_2$, ... has the transition probability matrix
$$
P = \left[
\matrix
{
0.3&0.2&0.5 \\
0.5&0.1&0.4 \\
0&0&1
}
\right]
$$
and is known to ...
1
vote
1answer
51 views
Symmetry of hitting times in a Markov chain
Consider an irreducible, aperiodic Markov chain with stationary distribution $\pi$. We will use $E_{\pi} T_j$ to be the hitting time of node $j$ when the initial distribution is $\pi$, and $E_i T_j$ ...
3
votes
0answers
82 views
maximum renewal rate of a Markov chain
Consider a Markov chain $(X_t)$ on a state-space with a countably generated $\sigma$-algebra and assume this Markov chain allows for small sets of order one. This means there exist sets $\mathfrak{S}$ ...
2
votes
0answers
17 views
Problem with the uniform transience
Let $X$ be a Borel space and let us consider a Markov Chain $(\Phi_n)_{n\geq 0}$ on this space given by the stochastic kernel
$$
P(x,\mathrm dy) = p(x,y)\mu(\mathrm dy)
$$
where the density $p$ is ...
5
votes
0answers
53 views
Confusion in the proof of properties for $\psi$-irreducibility
Let $P$ be a stochastic kernel on a measurable space $(\mathsf X,\mathfrak B(\mathsf X))$. The kernel $P$ is called $\varphi$-irreducible if for a positive measure $\varphi$ and for all measurable ...
0
votes
0answers
36 views
Markov Chain: Solidarity theorems
When a chain is irreducible (so each state can be reached from every other state, eventually), we quote that all states have the same character: all aperiodic / periodic with the same period, all ...
1
vote
1answer
133 views
Unique Stationary Distribution for Reducible Markov Chaine
Is it possible for a reducible markov chain to have a unique stationary distribution. Consider e.g. the markov chain with transition matrix below
$$A=
\begin{pmatrix}
1 & 0 & 0 \\\
0.2 ...
2
votes
1answer
85 views
Random walk with 3 possible steps
I have i.i.d. random variables with following distribution:
$$ P(\xi_i =1) = p_1, \ P(\xi_i = 0) = p_0, \ P(\xi_i = -1) = p_{-1}; \quad S_n = \sum^n_{i=1}\xi_i.$$
I am interested in probability of ...
0
votes
1answer
67 views
What is the expected number of point in time it is in room $2$
A man is forced at time $0$ into a five-room maze shown as the diagram. At the end of each unit of time, it changes to a different room by choosing an exit at random. Let $X_n$ be the room number ...
5
votes
1answer
100 views
A question about how to get the limiting probability.
Suppose $p=\begin{bmatrix}
0& 1\over 3 &0 &2\over 3 \\
0.3& 0& 0.7 &0 \\
0& 2\over 3&0 &1\over3 \\
0.8& 0& 0.2& 0
\end{bmatrix}$is the ...
0
votes
2answers
55 views
Proving $X$, $Y$, $g(Y)$ is a Markov Chain in That Order
I wondering how to prove $X$, $Y$, $g(Y)$ is a Markov Chain in That Order?
$X$, $Y$, $Z$ is a Markov Chain in That Order (denoted $X\to Y\to Z$) if $$p(x,y,z) = p(x)\cdot p(y\mid x)\cdot p(z\mid ...
0
votes
1answer
65 views
Transition probability convergence for Harris Chains
I'm studying theorem 6.8.8 of Durrett - convergence of transition probabilities for Harris chains and I have a (I think) pretty hard question which would help me more than words can describe if one of ...
0
votes
0answers
33 views
Does a Markov chain have to be adapted
My definition of Markov is a priori
$P(X_{n+1}\in A|\mathcal{F}_n)=P(X_{n+1}\in A|X_n)$
I want to assume that $\mathcal{F}_n$ is the natural filtration $\sigma (X_1,\dots ,X_n)$, but I know that the ...
3
votes
1answer
175 views
Countable state Markov chain: detailed balance consequences
Let $S$ be a countable set and $\pi$ a probability distribution on $S$. A discrete-time
Markov chain $(X_n)$ with state space $S$ is said to be in detailed balance
with respect to $\pi$ (or simply in ...
1
vote
0answers
51 views
An Iterated function system with probabilities and overlapping supports of its invariant measures
Let $(X, \rho)$ be a Polish space. Consider an Iterated Function System $(S_i,p_i)_{i=1,...,N}$, where $S_i:X\rightarrow X$, $p_i: X\rightarrow \left[0,1\right]$ are continuous functions and ...
1
vote
2answers
79 views
Specific question to a Markov chain proof in Durrett
I apologize if this is to specific but i've already talked to two of my professors without much success and I really need to understand this subject. The following theorem is stated in Durrett page ...
1
vote
1answer
127 views
Constructing a discrete Markov chain
Klenke gives a construction for a discrete Markov chain (Section 17.2 "Discrete Markov Chains: Examples", pp. 353-354). I don't understand several points in this construction, as indicated below.
The ...
1
vote
1answer
75 views
Why is the Markov property implied by the existence of a transition matrix?
If $\left(X_n\right)_{n\in\mathbb{N}_0}$ is an $E$-valued stochastic process with distributions $\left(P_x\space:\space x\in E\right)$ satisfying
$$\mathrm{P}_x\left(X_0=x\right)=1$$
and stochastic ...
0
votes
0answers
54 views
Martingale with reflecting barrier
I am not very familiar with the theory of martingales or random walks, perhaps someone could point me in the right direction or give me some help with the following problem.
Consider a random ...
0
votes
1answer
200 views
Why is a random walk a time-homogeneous Markov process?
Why is a random walk on $\mathbb{R}^d$ (see below) a time-homogeneous Markov process? Specifically, why does it satisfy requirement #2 of definition 17.3 that the map ...
1
vote
1answer
123 views
Transforming an inhomogeneous Markov chain to a homogeneous one
I fail to understand Cinlar's transformation of an inhomogeneous Markov chain to a homogeneous one. It appears to me that $\hat{P}$ is not fully specified. Generally speaking, given a $\sigma$-algebra ...
1
vote
0answers
40 views
Markov chain supplementary litterature
I'm studying markov chains through Durrett and I'm finding it quite hard to read.
Does anyone have a good idea to supplementary book, preferably one on his level of generality and which studies some ...
0
votes
1answer
153 views
How are the pairs of two independent pure-birth processes a Markov process?
A pure-birth Process is a generalization of a homogeneous Poission process. Whereas in the Poisson process the holding times between jumps are iid exponentially distributed random variables with ...
0
votes
1answer
73 views
The existence of stopping rule from one distribution to another.
Let $(X_n, n \ge 0)$ be a Markov chain. Let $V$ be the state space. Let $\lambda$ and $\tau$ be two probability distribution. Can we say that for any $\lambda$ and $\tau$, there is always a ...
1
vote
1answer
175 views
A Markov chain probability calculation.
I'm taking a course about Markov chain, and here's a snippet from the lecture notes:
Let $(X_i, i \ge 0)$ be a time homogeneous Markov chain, let $V$ be
the state space, let $\lambda$ be the ...
0
votes
1answer
154 views
Conditional Expectation.
Are the following two the same:
$E[V(X_{t_{k+1}})|g(X_{t_{k+1}}),X_{t_k}]$
and
$E[E[V(X_{t_{k+1}})|g(X_{t_{k+1}})]|X_{t_k}]$
Where $X$ is Markov chain
$X_{t_k} \in \mathcal{R}^n$
$V: ...
2
votes
2answers
116 views
Markov chain basic positive recurrency question
If a discrete markov chain is stationary (as far as I know: doesn't modify itself with time), irreducible (doesn't have transient states) and aperiodic (no periodic states), is it positive recurrent?
...
2
votes
0answers
139 views
Computing the stationary distribution of a markov chain
I have a markov chain with transition matrix below,
$$\begin{bmatrix}
1-q & q & & & \\
1-q & 0 & q & & \\
& 1-q & ...
1
vote
2answers
95 views
Finding again the stationary distribution of a markov chain
I am asked to compute the stationary distribution of the markov chain with state space $E=\{0\dots,n\}$ and transition matrix below:
\begin{bmatrix}
0 & 1 \\
\frac{1}{n} ...
3
votes
1answer
506 views
Finding the stationary distribution of a markov chain
I am asked to compute the stationary distribution of the markov chain with state space $E=\mathbb{N}_0$, $q_n >0$ for all $n \in \mathbb{N}_0$ and transition matrix below:
\begin{bmatrix}
q_0 ...
2
votes
1answer
467 views
Calculating stationary distribution of markov chain
I am asked to compute the stationary distribution of the markov chain with state space $E=\{0\dots,n\}$ and transition matrix below:
\begin{bmatrix}
0 & 1 \\
\frac{1}{n} ...
2
votes
1answer
1k views
Kendall notation's “General distribution”, what does that mean?
The first and second parameters for the Kendall's notation may have a G value, which stands for General distribution, see here.
But what does that mean? What is a ...
5
votes
1answer
544 views
Stationary distribution of random walk
Let $\mathcal{X}$ be a simple random walk with barrier at zero, state space $E = \mathbb{N}_0$ and transition matrix below with $0<q<1$.
\begin{bmatrix}
1-q & q & & ...
1
vote
1answer
820 views
Show irreducibility of markov chain
I need to show that the markov chain that has transition matrix written below is irreducible.
\begin{bmatrix}
0.2 & 0.5 & 0.1 & 0.1 & 0.1 \\
0.2 & 0.5 ...
2
votes
1answer
129 views
Confused about Markov property
The sample space is $\Omega$ with $\omega = (\omega_0, \omega_1, \ldots) \in \Omega$ an infinite sequence of a set $S$. So the measure space is $(S^{\mathbb{N}}, \mathcal{S}^{\mathbb{N}})$ where ...
1
vote
0answers
92 views
Continuous-time Markov process - need help with flux and discrete-time equivalent
I have a continuous-time markov process and I need to calculate the following
transition frequencies matrix (aka intensity matrix)
transition probabilities
all parameters which define permanence ...
3
votes
1answer
322 views
Strong Markov property - Durrett
I recently had great success with my first question here so I will boldly go on to a second.
Here goes:
I'm studying Markov Chains in Rick Durrett - Probability: Theory and example and I'm stuck with ...
1
vote
1answer
175 views
Finite State Markov Chain Stationary Distribution
How does one show that any finite-state time homogenous Markov Chain has at least one stationary distribution in the sense of $\pi = \pi Q$ where $Q$ is the transition matrix and $\pi$ is the ...
0
votes
1answer
216 views
Find stationary distribution decomposable Markov chain
Again a probability exercise:
Let $X=U \cup V$ be the finite state space of a Markov chain, where $U$ and $V$ are disjoint subsets of $X$ and $p_{ij}=0$ if both $i,j \in U$ or both $i,j \in V$. ...
