Skip to main content

Questions tagged [markov-process]

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

Does is hold that $E[f(X_t)1_{\{ s \geq T_1\}}| \mathcal F_{T_1}] = P_{t-T_1}(X_{T_1})1_{s \geq T_1}$ for $s\leq t$ and X is a strong Markov process

Let $X=(X_t)_{t\geq 0}$ be a homogeneous cadlag Markov process taking values in a finite state space $S$. Let $T_1$ be its first jump time and $f$ be a bounded measurable function. I would like to ...
mathnoob's user avatar
  • 149
0 votes
0 answers
26 views

convergence of Poisson equations

Let $E$ be a locally compact separable metric space and $B(E)$ be a Banach space of bounded measurable functions on $E$. For each $n \in \mathbb{N} $, $L^n$ and $L$ are linear operators on $B(E)$ ...
FileHandler's user avatar
0 votes
1 answer
37 views

Markov versus Martingale

My text provides an illuminating example about the fact that Markov processes are not martingales in general and martingales are not Markov processes in general. First, the standard brownian motion $(...
Quasar's user avatar
  • 5,450
0 votes
0 answers
12 views

Is there a way to measure interdependence of steps in diffusion/reverse diffusion process? [closed]

I know that that diffusion process is Markovian in theory, in that the consecutive steps do not depend on each other. Is there a way to draw some predictions (with measurable uncertainty) for the next ...
Yeshwanth Venkatesha's user avatar
0 votes
0 answers
22 views

$\mathbb P_{x_0}(\tau>t\mid X_1=x_1)=\mathbb P_{x_0}(\tau>t)$

Let $(\Omega,F,\mathbb P_x,(Y_t)_{t\geq 0})_{x\in E}$ be a continuous time Markov chain with countable state space $E$. I found the following identity on page 73 of Liggett's introduction to ...
Filippo's user avatar
  • 3,584
3 votes
1 answer
44 views

Levy Process is a Feller Processs

I want to show that every Levy Process is a Feller Process. Let $ X=(X_t)_{t\geq 0}$ be a Levy Process and $\mu_t(dy):=P\circ X^{-1}(dy)$ . I found a proof where it is shown that the transition ...
kays44's user avatar
  • 81
1 vote
0 answers
45 views

Is there a notion of approximation of continuous-time Markov processes by finite-valued Markov processes?

Recall that in practice, to simulate a Brownian motion on $[0,1]$, we usually use the interpolated process $X^n=(X^n_t)_{t\in[0,1]}$ between the jumps of a random walk $(S_k)_{k=1,...,n}$ with $n$-...
Jeffrey Jao's user avatar
2 votes
1 answer
48 views

A cadlag Feller process for $\mathcal F$ is Markov w.r.t $\mathcal F_+$ (Th. 46, Chap. 1, Stochastic Integration - Protter)

In page 35 of the book Stochastic Integration by P. Protter, he defines a Feller process as follows: Then he states the following theorem. In the proof, he used the following strategy: Next, he ...
Jeffrey Jao's user avatar
2 votes
0 answers
28 views

Tail Probability of r.v. $X_t$ from a Langevin diffusion

Backgrounds: An overdamped Langevin dynamics on $\mathbb{R}$ is defined as the solution to the following SDE: $$ dX_t=-\nabla V(X_t)\,dt+\sqrt{2\beta^{-1}}\,dB_t,\qquad X_0=x_0. $$ If $V(x)=\frac{x^2}{...
Hirofumi Shiba's user avatar
1 vote
0 answers
33 views

Optimal transition matrix that minimizes mixing time

For a finite state space, say I have a transition matrix $P$ such that it is irreducible and aperiodic. The stationary distribution is $\pi$. I wonder if there's any literature on how to pick an extra ...
Kryvtsov's user avatar
  • 131
-1 votes
0 answers
78 views

Random walk with indipendent but not identically distributed increments

Suppose $\{Z_i\}_{i=1,2, \ldots}$ are normally distributed (independent but not identically distributed) random variables with mean $\mu(y)>0$ and positive variance $\sigma^2(y)$. Therefore we ...
ag_c1768918's user avatar
2 votes
1 answer
75 views

Definition of a Markov process

I found 2 Definitions for a Markov process and I am trying to understand how they are connected. Let $X=\left(X_t\right)_{t\geq 0}$ be a $\mathcal{F}_t$ adapted Process. We say $X$ is a Markov ...
kays44's user avatar
  • 81
1 vote
0 answers
84 views

Fundamental theorem of Markov chains for integrable functions

Let $(X_n)_{n\in \mathbb{N}}$ be an irreducible, positive recurrent and aperiodic Markov chain on a countable set $E$, with stationary distribution $\pi$. Let $f\in L^1(\pi)$. Is it true that $P^nf(x):...
No-one's user avatar
  • 667
1 vote
1 answer
53 views

How do we use strong Markov property to get the formula

Suppose $X(t)$ is a homogeneous Markov process and satisfies the strong Markov property. Assume that $U\subset\mathbb{R}^{n}$ is an open and bounded set, denote the boundary of $U$ by $\Gamma$, $\tau_{...
R-CH2OH's user avatar
  • 351
0 votes
1 answer
34 views

Markov Chain Detailed Balance $\pi(x)*P(x, y) = \pi(y)*P(y, x)$

Let's say I have a Markov chain and it has a transition matrix denoted as $P$. The $(row, column)$ elements of the $P$ matrix are denoted as $P(i, j)$. Just by looking at the transition matrix $P$, ...
PredsFan13's user avatar
1 vote
0 answers
41 views

Is a randomly restarted Markov process again Markov?

Let $E$ be a topological space and $(X_t)_{t\ge0}$ be a right-continuous time-homogeneous Markov process with transition semigroup $(P_t)_{t\ge0}$ on a filtered probability space $(\Omega,\mathcal A,(\...
0xbadf00d's user avatar
  • 13.9k
0 votes
0 answers
33 views

Proof that excessive function are also regular (super-harmonic)

In page 117 of Shiryaev's book optimal stopping rules, he claims that the excessive functions are also regular under some condition and state the proof is analogous to another proof, but I fail to ...
Stocavista's user avatar
0 votes
0 answers
16 views

Model probabilistic time evolution from vector field distribution

Given an axis-aligned grid${}^1$ $x_1 , \ldots , x_n \in \mathbb{R}^2$. Let $\mu \in \mathbb{R}^{2n}$ and $\Sigma \in \mathbb{R}^{2n\times 2n}$ be the means and covariance matrix and assume we have a ...
Targon's user avatar
  • 1,114
0 votes
1 answer
69 views

What are the stationary distributions of this Markov Chain?

The Transition Matrix of the chain is : $$ \begin{bmatrix} 0 & 1/3 & 2/3 & 0 & 0 & 0 & 0 & 0 \\ 2/3 & 0 & 1/3 & 0 & 0 & 0 & 0 & 0 \\ 1/3 & 0 ...
Steve M.'s user avatar
0 votes
1 answer
34 views

For which values of $a$ is this Markov Chain irreducible and aperiodic?

The transition matrix is: $$ \begin{bmatrix} a & 1-a & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0\\ 1/2 & 0 & 0 & 0 & 1/2 \\ 0 & 0 & 1 & 0 & 0 \\ 0 &...
Steve M.'s user avatar
3 votes
0 answers
86 views

Identify if the following are Markov processes

I'm taking the MIT 18.S096 2013 OCW and stumbled across the following problem: identify whether the following are Markov processes: (d) $X_0 = 0$ and $X_{t+1} = X_t+ Z_tZ_{t-1} \cdot Z_0$ for $t\geq0$,...
calgary-enjoyer's user avatar
2 votes
2 answers
96 views

If $X$ is a time-homogeneous Markov process, is $\tilde X:=X-\lfloor X\rfloor$ Markov as well?

Assume $(X_t)_{t\ge0}$ is an $\mathbb R^d$-valued time-homogeneous Markov process with transition semigroup $(\kappa_t)_{t\ge0}$ on a probability space $(\Omega,\mathcal A,\operatorname P_x)$ ...
0xbadf00d's user avatar
  • 13.9k
3 votes
0 answers
35 views

Expectation value in Markov process using backward generator and semigroup

For the past few days, I've been studying continuous time Markov processes. I was making some exercises and ran into trouble with the following: Consider a continuous Markov process. The state space ...
luki luk's user avatar
  • 189
1 vote
1 answer
47 views

Decomposing a general stopping time into stopping components

Let $(X_n)_{n \geq 0}$ be a discrete-time Markov chain taking values in a finite state space $S$, with transition matrix $P$. Let $(\mathcal F_n)_{n\geq 0}$ be the natural filtration and let $\tau \...
Jeffrey Jao's user avatar
0 votes
0 answers
24 views

Regarding 1D Asymmetric Simple Exclusion Processes

I have been trying to decipher a paper on Asymmetric Simple Exclusion Processes in 1D by B. Derria: "An exactly soluble non-equilibrium system: The asymmetric simple exclusion process". ...
mathphyguy's user avatar
2 votes
1 answer
51 views

Finding References for known results about mixing time of a Markov chain arising from Gaussian elimination

I am looking for references for the known results on the following problem. Let $(X_t)$ be a Markov chain on $GL_n(F_2)$, where in each step, an ordered pair $(i,j)$ is chosen uniformly at random, and ...
mathematico's user avatar
0 votes
3 answers
58 views

Is $(tW_{t^2})$ a Markov Process?

As in the question, is $X_t = tW_{t^2}$ Markov process with respect to the filtration generated by $X$? My attempt. Filtration generated by $X_t$, so $\sigma(X_s, s \le t)$, is the same as the ...
Barabara's user avatar
  • 704
0 votes
0 answers
18 views

Hitting Times by Translation Invariant Processes with Asymptotic Velocity

I am trying to prove the following statement which seems to me intuitive, but I can't work out how to formalise it. Suppose you have a stochastic process $(X(t))_{t\geq 0}$ on $\mathbb{R}^d$ which is ...
user1598's user avatar
  • 395
1 vote
0 answers
42 views

Existence of Fokker-Planck equation under Cauchy boundary condition.

A 1D Fokker-Planck equation within a constrain region is uniquely characterized by three functions and a boundary conditon: A drifting term $\mu(x,t)$. A diffusion term $\sigma(x,t)$. An initial ...
陈进泽's user avatar
  • 133
1 vote
0 answers
29 views

At what values ​of $p$ will the chain be recurrent?

Consider a Markov chain on $\mathbb{Z}$ with $p_{i,i+2} = p$ and $p_{i,i−1} = 1 − p$. At what values ​​of $p$ will the chain be recurrent? I know that a chain is recurrent if for any state $j$ $$\sum\...
ABlack's user avatar
  • 582
3 votes
1 answer
65 views

When is an ergodic Ito diffusion also mixing

If $(X_t,t\geq0)$ denotes the strong solution to the Ito SDE $\mathrm{d}X_t=b(X_t)\mathrm{d}t+\sigma(X_t)\mathrm{d}W_t$ for $W_t$ the standard $d$-dimensional Brownian motion and $b,\sigma$ satisfying ...
Daan's user avatar
  • 362
0 votes
0 answers
32 views

Are ergodic continuous time processes (strictly) stationary in the limit?

If $X_t$ is a continuous time Markov process in a general state space $X$, say $X=\mathbb{R}^d$. Is it necessarily true that $X_t$ is stationary in the limit, as I believe that any ergodic process ...
Daan's user avatar
  • 362
0 votes
1 answer
79 views

Probability of $r$ consecutive heads occurring exactly once in infinite tosses

I was thinking about this problem: In an infinite number of tosses, what is the probability of exactly $r$ consecutive heads exactly once if the probability of a head in a single toss is $p$? My ...
Ryukendo Dey's user avatar
1 vote
0 answers
18 views

Is there a Dirichlet form on product space?

Let $(X_t)_{t\ge 0}$ be a Markov process on a locally compact Polish space $E$ (say for example the Sierpinski Gasket). Then there is a Dirichlet form associated with $X_t$ on $E$, call it $({\cal E},{...
JY0's user avatar
  • 113
0 votes
0 answers
16 views

Weak ergodicity of discrete birth death process

I am trying to understand the criteria of weak ergodicity through the Dobrushin coeffcient. Let $P(m, k)$ be a transition matrix of a countable state space discrete time Markov chain. Weak ergodicity ...
Koncopd's user avatar
  • 995
0 votes
0 answers
16 views

Brownian motion is Locally Holder contiuous $ |B(t)-B(s)| \leq C|t-s|^\alpha, \quad 0 \leq s, t \leq N, $

The following is taken from Exercise 22 in section 1.5 of Ligget's continuous time markov processes. It states that Use Doob's inequality, Theorem A.42, applied to even powers larger than the fourth ...
Math_Day's user avatar
  • 1,255
0 votes
0 answers
16 views

Continuous time Markov Processes, transition rate matrix Q

I have these lecture notes on continuous time Markov Processes: CTMPs I don't seem to understand the relation $Q=P'(0)$, for P: the transition probability matric and Q: the transition rate matrix. ...
martyna1222's user avatar
0 votes
0 answers
25 views

Counting arborescences in time-inhomogeneous stationary markov chain

Problem: Consider a directed graph G which is irreducible i.e. there is a directed path from any node to any node. For each $i$, let $S(i):=\{j:i\to j\}$ and $|S(i)|:=$ the cardinality of S(i). Let $\{...
Kumar's user avatar
  • 1,167
0 votes
0 answers
34 views

Is the convergence speed of local time in Markov processes a continuum.

By theorem 3.12 (p. 427) in the book Continuous Martingales and Brownian Motion by Revuz and Yor, they claim that for an Harris recurrent Markov process $X$ with invariant measure $\mu$, and two ...
Daan's user avatar
  • 362
1 vote
0 answers
28 views

When is a $\Delta$-skeleton irreducible for diffusion processes?

Say we are given a diffusion process $X$ on $\mathbb{R}^n$, that is positive Harris recurrent. That is, for any measurable set $A$ with $\tau_A=\inf\{t\geq 0:X_t\in A\}$ and some $\sigma$-finite ...
Daan's user avatar
  • 362
2 votes
0 answers
84 views

Probability distribution of stochastic process with independent increments asymptotically approaching Gaussian

I am independently working through van Kampen's Stochastic Processes in Physics and Chemistry. It's a rewarding book to work through, but I am having trouble with question 46 on pg. 89. Below is the ...
Bulworth's user avatar
0 votes
1 answer
15 views

Why is the condition $\mathbf{P}\left(H_{t-1} \cap\left\{X_t=x\right\}\right)>0$ at the definition of markov property

In the book of David A. Levin, it states that A sequence of random variables $\left(X_0, X_1, \ldots\right)$ is a Markov chain with state space $\mathcal{X}$ and transition matrix $P$ if for all $x, ...
Our's user avatar
  • 7,337
1 vote
0 answers
45 views

Positive Harris recurrence and ergodicity for continuous time diffusion processes

For a diffusion process $X=(X_t)_{t\geq0}$ satisfying the usual conditions, we say that $X$ is Harris recurrent if, for a measurable set $A$, the first hitting time $\tau_A=\inf\{t\geq 0:X_t\in A\}$ ...
Daan's user avatar
  • 362
0 votes
0 answers
18 views

Stationarity vs. ergodicity of generalized Kesten process

Not an expert here. I am trying to figure out conditions that guarantee (i) existence of a stationary distribution, and (ii) ergodicity for the following generalized discrete-time Kesten process \...
Harry's user avatar
  • 21
0 votes
0 answers
8 views

Show that $m\ge 1$ $P_{i}(X_n=j \text{ for atleast distinct m integers n})=f_{ij}(f_{ji})^{m-1}$

Define $f_{ij}^{(n)}=P_i(X_n=j,X_t\ne j, 1\le t\le n)$ and $f_{ij}=\sum f_{ij}^{(n)}$ Show that $m\ge 1$ $P_{i}(X_n=j \text{ for atleast distinct m integers n})=f_{ij}(f_{ji})^{m-1}$ Can I get a ...
Raheel's user avatar
  • 1,711
0 votes
1 answer
20 views

Prove that $P(X_{n+m}=j|X_0=i_0,\dots, X_n=i)=p_{ij}^{(m)}$

Let $\{X_n\}$ satisfy the markov property. That is, $$P(X_{n+1}=j|X_0=i_0,\dots X_n=j)=P(X_{n+1}=j|X_n=j)=p_{ji}.$$ Then prove that $P(X_{n+m}=j|X_0=i_0,\dots, X_n=i)=p_{ij}^{(m)}$. Intuitively, this ...
Raheel's user avatar
  • 1,711
3 votes
1 answer
44 views

A question about Markov chain's definition

Background: One version of the definition of a discrete-time Markov chain is as follows: Let $S$ be a countable set. A stochastic process $(Z_n)_{n\in\mathbb{N}_0}$ is called a discrete-time Markov ...
Karl's user avatar
  • 33
2 votes
1 answer
23 views

Demonstrate a (continuous time) chain that increases by one, and then randomly returns to the origin, is transient

I have the following continuous time Markov chain $X = (X_t : t \geq 0)$ with generator matrix given by $g_{i,i+1} = \lambda_i$ for $i \geq 0$, $g_{i, 0} = \lambda_i \rho_i$ for $i > 0$, and $g_{ij}...
Featherball's user avatar
2 votes
0 answers
116 views

How can we derive this integral inequality?

Furthermore, let $(E,\mathcal E,\lambda)$ be a $\sigma$-finite measure space; $\mu$ be a probability measure on $(E,\mathcal E)$; $\zeta$ be a Markov kernel on $(E,\mathcal E)$; $\pi$ be a ...
0xbadf00d's user avatar
  • 13.9k
4 votes
0 answers
53 views

Does a Poincaré inequality for a Markov process $X_t$ with invariant measure $μ$ infer a convergence rate of $\frac1t\int_0^tf(X_s){\rm d}s$ to $μf$?

Let $(X_t)_{t\ge0}$ be a time-homogeneous shift-ergodic Markov process with transition semigroup $(\kappa_t)_{t\ge0}$ and invariant measure $\mu$. One implication of a Poincaré inequality is a $L^2$-...
0xbadf00d's user avatar
  • 13.9k

1
2 3 4 5
53