1
vote
1answer
10 views

A certain formulation of the Chapman-Kolmogorov equation.

I am reading a book by Taira called Semigroups, Boundary Value Problems and Markov Processes. It is a nice read, but there is one thing I don't understand regarding the Chapman-Kolmogorov equation. A ...
0
votes
1answer
7 views

Markov Processes: $P_x$ and $E_x$

In the study of Markov processes, one usually introduces the measures $P_{\pi}$ on the path space of the process where $\pi$ is an initial distribution of the process $X$ i.e $\pi=\mathcal L(X_0)$. ...
0
votes
0answers
7 views

Equivalent Formulation of Markov Property for Homogeneous Chains

In Shiryaev's Probability (just above the strong Markov property, p.568), the author says that an equivalent formulation of the usual Markov property for homogeneous chains is $$P[\theta_nX\in B\mid ...
0
votes
0answers
12 views

Strong Markov Property for Discrete Stopping Times

I'm having a hard time deciphering a particular proof of the following strong Markov property. Theorem (Strong Markov Property) Let $X$ be a time homogenous Markov process with $T=\mathbb ...
0
votes
1answer
18 views

The “on $\left\{ \tau <\infty \right\}$” in the Strong Markov Property

The strong Markov property is often formulated as $$P[\theta _{\tau}X\in A\mid \mathscr F_{\tau}]\overset{\text {a.s on }\left\{ \tau <\infty \right\} }{=}P_{X_\tau}(X\in A)$$ What exactly does ...
0
votes
0answers
13 views

Expectation in Hidden markov process [on hold]

Let the hidden state variable be $S_t\in \{0,1\}$, and the observed variable be $X_t$. How can we prove that $\mathbb{E}(P(S_t=1|\sigma(X_{t-1},...,X_0)))= \frac{p_{01}}{p_{01}+p_{10}}$, where ...
2
votes
1answer
36 views

Markov processes on function spaces

Is there any reference on Continuous time Markov process whose state space is infinite dimensional function spaces, such as the space of continuous functions $C(R^d)$? It seems Dirichlet Form is a ...
1
vote
1answer
51 views

Markov Chains : Can anything be said about what happens in between two transition?

In time homogeneous discrete Markov chains we take a set period for a single transition. In examples we see sometimes depending on the examples the transition period being a a month a week etc. I'm ...
0
votes
3answers
80 views

Transition Matrix of M/M/1 Queue

We know that for an M/M/1 queue the state space is $S=\{0,1,2,... \}$. Further the probability to go from state $i$ to $i+1$ is $\lambda$ for all $i$ in $S$. Moreover, to go from $i$ to $i-1$ is the ...
0
votes
1answer
25 views

Markov property for a stochastic process with discrete state space.

Consider a stochastic process $\{X_s\}_{s\in\mathcal S\subseteq\mathbb R}$ with value in $(\mathbb R,\mathcal B(\mathbb R))$ adapted to a filtration $\{\mathcal F_s\}$ (we can suppose that ...
0
votes
1answer
43 views

Can the transition probabilities of an inhomogeneous Markov chain be written as an exponential?

If $Z_t$ is a homogeneous continuous-time Markov chain with finite state space $E=\{1,\ldots,p\}$, transition matrices $(P(t))$ and intensity matrix $Q$, it holds that $$ P(t) = \exp(tQ), $$ see for ...
3
votes
2answers
37 views

Always null recurrence at the boundary between positive recurrence and transience?

I have the following theorem: Let $\rho$ be the traffic intensity. a) If $\rho<1$, then $X$ is positive recurrent. b) If $\rho>1$, then $X$ is transient. c) If $\rho=1$, ...
1
vote
1answer
26 views

Discrete and Continuous Time Markov Properties

$\newcommand{\indep}{\perp\!\!\!\perp}$ In discrete time, the Markov property is $$P[X_{n+1}\in A\mid X_n=s_n,X_{n-1}=s_{n-1}\dots ]=P[X_{n+1}\in A\mid X_n=s_n]$$ On the other hand, the "general ...
1
vote
1answer
36 views

Q-matrix vs. P-matrix description of a Markov chain

Consider a continuous time Markov chain $(X_t)_{t \geq 0}$ on some state space $S$ with transition matrix (P-matrix) $p_t(x,y)$, the probability density of jumping from $x$ to $y$ in time $t$. The ...
1
vote
1answer
45 views

Property of Wiener process sample path

What is a mean of time, when the trajectory of wiener process $W_t$ is over the line $y=t$? We need to find $\mathbb{E}\tau$, where $\tau=\sum\limits_{a,b:\forall t\in(a,b) ; ...
1
vote
0answers
16 views

Branching Brownian Motion and KPP equation

I have troubles understanding the proof of the connection between BBM and KPP equation. I mean the proof of the next lemma from the lecture notes of Anton Bovier about BBM, link. This is almost whole ...
0
votes
0answers
16 views

A question in a textbook about Blumenthal 0-1 law for a general Markov process

This question came up as a result of reading this question . Here is the Blumenthal 0-1 law in the book Stochastic Processes by Richard F. Bass. Proposition 20.8 Let $(X_t , \Bbb{P}^x)$ be a ...
1
vote
1answer
60 views

random walk with sticky barriers

Consider a random walk on the line 1,...,d. You start at point 1. At each step you flip a coin: heads means go left, tails means go right. If you're at 1 and get a heads, just stay where you are ...
0
votes
1answer
20 views

Injections of Markov Processes

In office hours, a professor mentioned that an injective transformation of a Markov process remains Markov. Intuitively, this makes sense to me as you can "recover" the original Markov process from ...
0
votes
1answer
18 views

Markov Processes: How to show $\int P(X_t\in B\mid X_0)dPX_0^{-1}=P(X_t\in B)$?

Let $\left\{X_t \right\}_{t\in T}$ be a time homogeneous Markov process with state space $S$. How do I formally demonstrate$$P(X_t\in B)=\int_S P(X_t\in B\mid X_0)dPX_0^{-1}$$(here $PX_0^{-1}$ is the ...
0
votes
0answers
17 views

How to use symmetry of transition rate matrix in a continuous-time Markov chain?

This is part of a bigger question, so I have to change the question a bit to focus on the point. We have a continuous- time Markov chain with the following transition rate matrix: $$Q= \begin{pmatrix} ...
0
votes
0answers
55 views

Strong Markov property for Poisson point process

The question is thoroughly contained in the title. I just say that I would only like to find a reference for this question. I have searched in some books, to no avail. Just to avoid misunderstanding, ...
0
votes
1answer
19 views

A problem on Markov process

Suppose, $\Pi_{\theta}$ be the transition probability function of a Markov chain. For any function $f$ define $$\Pi_{\theta}f_{\theta}(x) = \int f(y,\theta)\Pi_{\theta}(x,dy).$$ Is there any ...
1
vote
1answer
33 views

Reflection Principle interpretation

Given a standard Brownian motion $(\Omega,\mathcal{F},(\mathcal{F}_t)_t,\mathbb{P},(B_t)_t)$ (the standard filtration $(\mathcal{F}_t)_t$), we define $$\forall t\ge 0: M_t:=\max_{0\le s\le t} B_s$$ ...
0
votes
2answers
52 views

Prove that something is a Markov chain

Let $\xi_0, \xi_1, \xi_2, ...$be independent, identically distributed, integer valued random variables. Define $Y_n$ = max{$\xi_i: 0 \leq i \leq n$}. Show that $(Y_{n)n\geq0}$ is a Markov chain and ...
1
vote
0answers
26 views

Analysis of Steady State Probability for Markov Process

I have a balance equation, representing a Markov Chain, which yields $$ (K - z) \pi(Z_c = z) = (\lambda_c + (z+1)x) \pi(Z_c = z+1) $$ where K is the maximum state of the server. The term $\lambda_c$ ...
1
vote
0answers
21 views

When are the marginals of an extremal invariant measure also extremal invariant?

Let's suppose that $X$ is a compact metric space, and thus as is $X \times X$. If given a Markov process on $X \times X$ with marginals that are Markov processes on $X$, then we know that the ...
1
vote
1answer
34 views

$X_n, n> 0$ is a Markov Chain, how to interpret $Z_n = (X_n,X_{n+1}), n > 0$?

Am a newbie to Markov Chain. So, this might be incredibly naive/stupid question. If $X_n, \, n > 0$ is MC, am having difficulty imagining/interpreting process $Z_n = (X_n,X_{n+1}), n > 0$. I ...
0
votes
0answers
51 views

Stationary distribution of a “birth-death model” that does not have Markov property

A typical birth-death process is defined such as the probability of going from any state $j$ to any state $i$ is given by: $$ p_{ij}= \begin{cases} b_i & \text {if $j = i+1$} \\ ...
2
votes
1answer
63 views

continuous time Markov chain, something the book does not explain

I have a problem with something in my book, under the chapter of continuous time Markov chains. I will post a link to what the book does. They do something which they seem to take for granted, but I ...
0
votes
1answer
48 views

Representation of Markov process adapted to given Filtration

Let $X$ be continuous Markov process adapted to a filtration generated by Brownian motion $B$. Does there exist a function $f$ such that $X_t = f(t,B_t)$? My guess is that it should have such ...
0
votes
0answers
63 views

Continuous time Markov chains, how would this definition be expanded from time-homogeneous to time-inhomogeneous.

Below I have a picture of how we can view a continuous time Markov chain that is time-homogeneous. Now, I am wondering what happens when we have a inhomogeneous continuous Markov chain. I have ...
0
votes
1answer
54 views

Markov processes and semimartingales

Semimartingales and Markov processes are two fundamental families in probability theory. There are many specific processes that belongs to the intersection of those two families, e.g. Levy processes. ...
9
votes
2answers
348 views

What is the importance of the infinitesimal generator of Brownian motion?

I have read that the infinitesimal generator of Brownian motion is $\frac{1}{2}\small\triangle$. Unfortunately, I have no background in semigroup theory, and the expositions of semigroup theory I have ...
1
vote
2answers
98 views

Expected number of steps between states in a Markov Chain

Suppose I am given a state space $S=\{0,1,2,3\}$ with transition probability matrix $\mathbf{P}= \begin{bmatrix} \frac{2}{3} & \frac{1}{3} & 0 & 0 \\[0.3em] \frac{2}{3} ...
0
votes
1answer
73 views

Exercise on Markov chain

Prove, or give an explicit counterexample to refute, the following assertion: if $\{X_n\}$ is a Markov chain, then $\{X_n^2\}$ is also a Markov chain. It's easy to show that ...
3
votes
1answer
71 views

Question about Markov chain

We know that if $\{X_n\}$ is a Markov chain, then $X_{n+1}$ is independent with the past states $X_0,\ldots,X_{n-1}$ given current state $X_n$, that is ...
3
votes
1answer
105 views

Markov chain problem with finite states

Consider any Markov chain on a state space with exactly $r$ states. We want to find the largest $N>0$ such that there exists states $i,j$ for every $n < N$ we have $p_{ij}^{(N)} > 0$ and ...
1
vote
0answers
37 views

Markov Chain depicting unruly customer behavior

A store has 2 bins of balls. 1 bin is red, and contains 3 red balls. The other bin is gray and contains 2 gray balls. Every minute, on the minute, exactly one customer comes by the bins, picks up ...
1
vote
1answer
66 views

Library chain stationary distribution

This is an exercise 1.47 from Richard Durrett's Essentials of Stochastic Processes p.85 (doi: 10.1007/978-1-4614-3615-7_1 or Google Books). On each request the ith of the $n$ possible books is the ...
4
votes
1answer
80 views

random walk in a certain environment

Consider the following random walk in one dimension, starting from $r(0)=0$. $$ r(i+1) = r(i) + \xi, $$ where $\xi(i, r(i))$ is an increment with distribution $P(\xi=1) = \frac{c^{r(i)}}{i-r(i)+1}$ ...
2
votes
0answers
48 views

A theorem in the paper “Noncommuting Random Products” by Furstenberg

I have a question concerning the proof of theorem 2.5 at page 395 of the paper Noncommuting Random Products, by H. Furstenberg, Trans. Amer. Math. Soc., 1963. The statement is as follows: Let $\mu$ ...
5
votes
1answer
114 views

Log-likelihood function

I'm not sure if this could be asked here, or in math overflow... In the following paper Cho, Jin Seo, and Halbert White. "Testing for regime switching." Econometrica 75.6 (2007): 1671-1720. doi: ...
2
votes
1answer
146 views

Equivalent defining Markov property

Consider the stochastic process $(X_t)_{t \in \mathbb{R}}$ and show the equivalence of the following two Markov properties: (a) $P(X_t \in A \mid X_u, u \leq s) = P(X_t \in A\mid X_s) \qquad ...
1
vote
1answer
53 views

Strong Markov property given transition functions

Suppose we are given family of transition functions satisfying Chapman-Kolmogorov equation, what conditions will ensure that there exists a continuous or cadlag Markov process with given transition ...
0
votes
1answer
58 views

Stationary distribution for a Markov chain which is not irreducible

I have a Markov chain with $K$ states $S$: {$s_1,s_2,...,s_K$}. $s_1$ is reachable from any state in $S$; however not all the states can be reached from $s_1$. What does the stationary distribution ...
0
votes
0answers
57 views

Continuous-Time Markov Chain conditioned on not visiting a part of the state space

Let $(X_t)_{t\geq 0}$ be a homogeneous continuous-time Markov chain with state space $\Psi =\{1,\dots,N\}$. Consider $S=\{1,\dots,M\} \subset \Psi$. Define $T = \inf \{ t \geq 0 : (X_t \not \in S ...
2
votes
1answer
82 views

Markov chain and hitting times

I have a Problem about hitting times. That's the following: Let $A\subset E$ and the first passage time $T_A$ and the hitting time $H_A$. Define: $T_A =\inf\{n\geq 0;X_n \in A\}$ and $H_A ...
2
votes
1answer
73 views

transition kernel

I've got some trouble with transition kernels. We look at Markov process with statspace $(S,\mathcal{S})$ and initial distribution $\mu^0$. We have a transition kernel $P:S\times ...
0
votes
0answers
83 views

Markov Chains Proof using Statistics

Source: This came from "Introduction to probability" by Charles Miller Grinstead, and James Aurie Snell. It was located on page 407 and is Theorem 11.1 in the section 11.1 Introduction. Theorem: The ...