1
vote
1answer
91 views

Brownian Motion inequality (related to Dvoretzky-Erdoes test)

i have the following question: Let $B(t)$ be a d-dimeansional Brownian motion $d\ge 3$, and $f$ be a monoton increasing function from the positive reals to the positive reals. Let $A_n=(\exists t\in ...
0
votes
1answer
16 views

Brownian Motion and Progressive Process

Let $B_t$ be a Brownian motion. Define sign function as follows. $sign(0) = 0$ and $sign(x) = \frac{x}{|x|}, \forall x \neq 0$. I do not know how to show the following two questions, especially on the ...
-1
votes
0answers
23 views

Stationary distribution of a birth-death model where a parameter follows a uniform distribution.

I asked this question about some type a markov process I was interested in. @Did offers an answer but I fail to understand how to apply his answer to a concrete example. I am therefore seeking for an ...
1
vote
0answers
48 views

Can we find a correlation between states of a Markov chain?

I have a fair bit of knowledge on Markov chains but I recently wondered if there is a way to find out a correlation between the states of a finite Markov chain. I could not find any material on this. ...
1
vote
0answers
10 views

Application of Strong Markov Property

Theorem SMP (Strong Markov Property) Let $X$ be a time homogenous Markov process with $T=\mathbb R_+$ or $\mathbb Z_+$ and let $\tau$ be a stopping time taking countably many values. Then ...
-1
votes
0answers
19 views

Conditional independence in Markov family. [closed]

Suppose that X , ($\Omega, \mathbb{F}$), ${\lbrace}{P^x}{\rbrace}_{x \in \mathbb{R}^d} $ is a markov family with shift operators ${\lbrace} \theta_s{\rbrace}_{s\geq 0}$. Using the fact that for every ...
0
votes
0answers
25 views

Covariance of states of a finite Markov chain

I know it is possible to construct a covariance matrix for states of a Markov chain but I cannot seem to find a proper way to compute it. I will attach some theories I found from Kemeny and Snell's ...
1
vote
0answers
83 views

Hidden Markov Model, transition probabilities which are modeled with an exponential distribution

I'm looking at implementing an algorithm described in a paper, but I'm having trouble understanding how the transition probabilities for a Hidden Markov Model are defined. In the first sections, I ...
0
votes
1answer
15 views

Recurrence of states in a function of a Markov chain

Suppose $X$ is a Markov chain (or process, for that matter) and suppose further $f(X)$ is also a Markov chain. Let $s$ be a recurrent state in $X$. Is there a general way to determine the recurrence ...
1
vote
1answer
18 views

A certain formulation of the Chapman-Kolmogorov equation.

I am reading a book by Taira called Semigroups, Boundary Value Problems and Markov Processes. It is a nice read, but there is one thing I don't understand regarding the Chapman-Kolmogorov equation. A ...
0
votes
1answer
11 views

Markov Processes: $P_x$ and $E_x$

In the study of Markov processes, one usually introduces the measures $P_{\pi}$ on the path space of the process where $\pi$ is an initial distribution of the process $X$ i.e $\pi=\mathcal L(X_0)$. ...
0
votes
0answers
9 views

Equivalent Formulation of Markov Property for Homogeneous Chains

In Shiryaev's Probability (just above the strong Markov property, p.568), the author says that an equivalent formulation of the usual Markov property for homogeneous chains is $$P[\theta_nX\in B\mid ...
0
votes
0answers
16 views

Strong Markov Property for Discrete Stopping Times

I'm having a hard time deciphering a particular proof of the following strong Markov property. Theorem (Strong Markov Property) Let $X$ be a time homogenous Markov process with $T=\mathbb ...
0
votes
1answer
19 views

The “on $\left\{ \tau <\infty \right\}$” in the Strong Markov Property

The strong Markov property is often formulated as $$P[\theta _{\tau}X\in A\mid \mathscr F_{\tau}]\overset{\text {a.s on }\left\{ \tau <\infty \right\} }{=}P_{X_\tau}(X\in A)$$ What exactly does ...
2
votes
1answer
36 views

Markov processes on function spaces

Is there any reference on Continuous time Markov process whose state space is infinite dimensional function spaces, such as the space of continuous functions $C(R^d)$? It seems Dirichlet Form is a ...
1
vote
1answer
55 views

Markov Chains : Can anything be said about what happens in between two transition?

In time homogeneous discrete Markov chains we take a set period for a single transition. In examples we see sometimes depending on the examples the transition period being a a month a week etc. I'm ...
0
votes
3answers
99 views

Transition Matrix of M/M/1 Queue

We know that for an M/M/1 queue the state space is $S=\{0,1,2,... \}$. Further the probability to go from state $i$ to $i+1$ is $\lambda$ for all $i$ in $S$. Moreover, to go from $i$ to $i-1$ is the ...
0
votes
1answer
30 views

Markov property for a stochastic process with discrete state space.

Consider a stochastic process $\{X_s\}_{s\in\mathcal S\subseteq\mathbb R}$ with value in $(\mathbb R,\mathcal B(\mathbb R))$ adapted to a filtration $\{\mathcal F_s\}$ (we can suppose that ...
0
votes
1answer
52 views

Can the transition probabilities of an inhomogeneous Markov chain be written as an exponential?

If $Z_t$ is a homogeneous continuous-time Markov chain with finite state space $E=\{1,\ldots,p\}$, transition matrices $(P(t))$ and intensity matrix $Q$, it holds that $$ P(t) = \exp(tQ), $$ see for ...
3
votes
2answers
38 views

Always null recurrence at the boundary between positive recurrence and transience?

I have the following theorem: Let $\rho$ be the traffic intensity. a) If $\rho<1$, then $X$ is positive recurrent. b) If $\rho>1$, then $X$ is transient. c) If $\rho=1$, ...
1
vote
1answer
27 views

Discrete and Continuous Time Markov Properties

$\newcommand{\indep}{\perp\!\!\!\perp}$ In discrete time, the Markov property is $$P[X_{n+1}\in A\mid X_n=s_n,X_{n-1}=s_{n-1}\dots ]=P[X_{n+1}\in A\mid X_n=s_n]$$ On the other hand, the "general ...
1
vote
1answer
39 views

Q-matrix vs. P-matrix description of a Markov chain

Consider a continuous time Markov chain $(X_t)_{t \geq 0}$ on some state space $S$ with transition matrix (P-matrix) $p_t(x,y)$, the probability density of jumping from $x$ to $y$ in time $t$. The ...
1
vote
1answer
47 views

Property of Wiener process sample path

What is a mean of time, when the trajectory of wiener process $W_t$ is over the line $y=t$? We need to find $\mathbb{E}\tau$, where $\tau=\sum\limits_{a,b:\forall t\in(a,b) ; ...
1
vote
0answers
19 views

Branching Brownian Motion and KPP equation

I have troubles understanding the proof of the connection between BBM and KPP equation. I mean the proof of the next lemma from the lecture notes of Anton Bovier about BBM, link. This is almost whole ...
0
votes
0answers
16 views

A question in a textbook about Blumenthal 0-1 law for a general Markov process

This question came up as a result of reading this question . Here is the Blumenthal 0-1 law in the book Stochastic Processes by Richard F. Bass. Proposition 20.8 Let $(X_t , \Bbb{P}^x)$ be a ...
1
vote
1answer
62 views

random walk with sticky barriers

Consider a random walk on the line 1,...,d. You start at point 1. At each step you flip a coin: heads means go left, tails means go right. If you're at 1 and get a heads, just stay where you are ...
0
votes
1answer
20 views

Injections of Markov Processes

In office hours, a professor mentioned that an injective transformation of a Markov process remains Markov. Intuitively, this makes sense to me as you can "recover" the original Markov process from ...
0
votes
1answer
18 views

Markov Processes: How to show $\int P(X_t\in B\mid X_0)dPX_0^{-1}=P(X_t\in B)$?

Let $\left\{X_t \right\}_{t\in T}$ be a time homogeneous Markov process with state space $S$. How do I formally demonstrate$$P(X_t\in B)=\int_S P(X_t\in B\mid X_0)dPX_0^{-1}$$(here $PX_0^{-1}$ is the ...
0
votes
0answers
19 views

How to use symmetry of transition rate matrix in a continuous-time Markov chain?

This is part of a bigger question, so I have to change the question a bit to focus on the point. We have a continuous- time Markov chain with the following transition rate matrix: $$Q= \begin{pmatrix} ...
0
votes
0answers
55 views

Strong Markov property for Poisson point process

The question is thoroughly contained in the title. I just say that I would only like to find a reference for this question. I have searched in some books, to no avail. Just to avoid misunderstanding, ...
0
votes
1answer
19 views

A problem on Markov process

Suppose, $\Pi_{\theta}$ be the transition probability function of a Markov chain. For any function $f$ define $$\Pi_{\theta}f_{\theta}(x) = \int f(y,\theta)\Pi_{\theta}(x,dy).$$ Is there any ...
1
vote
1answer
36 views

Reflection Principle interpretation

Given a standard Brownian motion $(\Omega,\mathcal{F},(\mathcal{F}_t)_t,\mathbb{P},(B_t)_t)$ (the standard filtration $(\mathcal{F}_t)_t$), we define $$\forall t\ge 0: M_t:=\max_{0\le s\le t} B_s$$ ...
0
votes
2answers
52 views

Prove that something is a Markov chain

Let $\xi_0, \xi_1, \xi_2, ...$be independent, identically distributed, integer valued random variables. Define $Y_n$ = max{$\xi_i: 0 \leq i \leq n$}. Show that $(Y_{n)n\geq0}$ is a Markov chain and ...
1
vote
0answers
28 views

Analysis of Steady State Probability for Markov Process

I have a balance equation, representing a Markov Chain, which yields $$ (K - z) \pi(Z_c = z) = (\lambda_c + (z+1)x) \pi(Z_c = z+1) $$ where K is the maximum state of the server. The term $\lambda_c$ ...
1
vote
0answers
21 views

When are the marginals of an extremal invariant measure also extremal invariant?

Let's suppose that $X$ is a compact metric space, and thus as is $X \times X$. If given a Markov process on $X \times X$ with marginals that are Markov processes on $X$, then we know that the ...
1
vote
1answer
38 views

$X_n, n> 0$ is a Markov Chain, how to interpret $Z_n = (X_n,X_{n+1}), n > 0$?

Am a newbie to Markov Chain. So, this might be incredibly naive/stupid question. If $X_n, \, n > 0$ is MC, am having difficulty imagining/interpreting process $Z_n = (X_n,X_{n+1}), n > 0$. I ...
0
votes
0answers
60 views

Stationary distribution of a “birth-death model” that does not have Markov property

A typical birth-death process is defined such as the probability of going from any state $j$ to any state $i$ is given by: $$ p_{ij}= \begin{cases} b_i & \text {if $j = i+1$} \\ ...
2
votes
1answer
64 views

continuous time Markov chain, something the book does not explain

I have a problem with something in my book, under the chapter of continuous time Markov chains. I will post a link to what the book does. They do something which they seem to take for granted, but I ...
0
votes
1answer
49 views

Representation of Markov process adapted to given Filtration

Let $X$ be continuous Markov process adapted to a filtration generated by Brownian motion $B$. Does there exist a function $f$ such that $X_t = f(t,B_t)$? My guess is that it should have such ...
0
votes
0answers
64 views

Continuous time Markov chains, how would this definition be expanded from time-homogeneous to time-inhomogeneous.

Below I have a picture of how we can view a continuous time Markov chain that is time-homogeneous. Now, I am wondering what happens when we have a inhomogeneous continuous Markov chain. I have ...
0
votes
1answer
59 views

Markov processes and semimartingales

Semimartingales and Markov processes are two fundamental families in probability theory. There are many specific processes that belongs to the intersection of those two families, e.g. Levy processes. ...
10
votes
2answers
438 views

What is the importance of the infinitesimal generator of Brownian motion?

I have read that the infinitesimal generator of Brownian motion is $\frac{1}{2}\small\triangle$. Unfortunately, I have no background in semigroup theory, and the expositions of semigroup theory I have ...
1
vote
2answers
110 views

Expected number of steps between states in a Markov Chain

Suppose I am given a state space $S=\{0,1,2,3\}$ with transition probability matrix $\mathbf{P}= \begin{bmatrix} \frac{2}{3} & \frac{1}{3} & 0 & 0 \\[0.3em] \frac{2}{3} ...
0
votes
1answer
74 views

Exercise on Markov chain

Prove, or give an explicit counterexample to refute, the following assertion: if $\{X_n\}$ is a Markov chain, then $\{X_n^2\}$ is also a Markov chain. It's easy to show that ...
3
votes
1answer
73 views

Question about Markov chain

We know that if $\{X_n\}$ is a Markov chain, then $X_{n+1}$ is independent with the past states $X_0,\ldots,X_{n-1}$ given current state $X_n$, that is ...
3
votes
1answer
107 views

Markov chain problem with finite states

Consider any Markov chain on a state space with exactly $r$ states. We want to find the largest $N>0$ such that there exists states $i,j$ for every $n < N$ we have $p_{ij}^{(N)} > 0$ and ...
1
vote
0answers
40 views

Markov Chain depicting unruly customer behavior

A store has 2 bins of balls. 1 bin is red, and contains 3 red balls. The other bin is gray and contains 2 gray balls. Every minute, on the minute, exactly one customer comes by the bins, picks up ...
1
vote
1answer
70 views

Library chain stationary distribution

This is an exercise 1.47 from Richard Durrett's Essentials of Stochastic Processes p.85 (doi: 10.1007/978-1-4614-3615-7_1 or Google Books). On each request the ith of the $n$ possible books is the ...
4
votes
1answer
80 views

random walk in a certain environment

Consider the following random walk in one dimension, starting from $r(0)=0$. $$ r(i+1) = r(i) + \xi, $$ where $\xi(i, r(i))$ is an increment with distribution $P(\xi=1) = \frac{c^{r(i)}}{i-r(i)+1}$ ...
2
votes
0answers
48 views

A theorem in the paper “Noncommuting Random Products” by Furstenberg

I have a question concerning the proof of theorem 2.5 at page 395 of the paper Noncommuting Random Products, by H. Furstenberg, Trans. Amer. Math. Soc., 1963. The statement is as follows: Let $\mu$ ...