# Questions tagged [markov-process]

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.

2,605 questions
Filter by
Sorted by
Tagged with
18 views

### Does is hold that $E[f(X_t)1_{\{ s \geq T_1\}}| \mathcal F_{T_1}] = P_{t-T_1}(X_{T_1})1_{s \geq T_1}$ for $s\leq t$ and X is a strong Markov process

Let $X=(X_t)_{t\geq 0}$ be a homogeneous cadlag Markov process taking values in a finite state space $S$. Let $T_1$ be its first jump time and $f$ be a bounded measurable function. I would like to ...
• 149
26 views

### convergence of Poisson equations

Let $E$ be a locally compact separable metric space and $B(E)$ be a Banach space of bounded measurable functions on $E$. For each $n \in \mathbb{N}$, $L^n$ and $L$ are linear operators on $B(E)$ ...
• 233
37 views

1 vote
33 views

### Optimal transition matrix that minimizes mixing time

For a finite state space, say I have a transition matrix $P$ such that it is irreducible and aperiodic. The stationary distribution is $\pi$. I wonder if there's any literature on how to pick an extra ...
• 131
78 views

### Random walk with indipendent but not identically distributed increments

Suppose $\{Z_i\}_{i=1,2, \ldots}$ are normally distributed (independent but not identically distributed) random variables with mean $\mu(y)>0$ and positive variance $\sigma^2(y)$. Therefore we ...
75 views

### Definition of a Markov process

I found 2 Definitions for a Markov process and I am trying to understand how they are connected. Let $X=\left(X_t\right)_{t\geq 0}$ be a $\mathcal{F}_t$ adapted Process. We say $X$ is a Markov ...
• 81
1 vote
84 views

• 351
34 views

### Markov Chain Detailed Balance $\pi(x)*P(x, y) = \pi(y)*P(y, x)$

Let's say I have a Markov chain and it has a transition matrix denoted as $P$. The $(row, column)$ elements of the $P$ matrix are denoted as $P(i, j)$. Just by looking at the transition matrix $P$, ...
1 vote
41 views

• 603
24 views

### Regarding 1D Asymmetric Simple Exclusion Processes

I have been trying to decipher a paper on Asymmetric Simple Exclusion Processes in 1D by B. Derria: "An exactly soluble non-equilibrium system: The asymmetric simple exclusion process". ...
51 views

### Finding References for known results about mixing time of a Markov chain arising from Gaussian elimination

I am looking for references for the known results on the following problem. Let $(X_t)$ be a Markov chain on $GL_n(F_2)$, where in each step, an ordered pair $(i,j)$ is chosen uniformly at random, and ...
• 133
58 views

### Is $(tW_{t^2})$ a Markov Process?

As in the question, is $X_t = tW_{t^2}$ Markov process with respect to the filtration generated by $X$? My attempt. Filtration generated by $X_t$, so $\sigma(X_s, s \le t)$, is the same as the ...
• 704
18 views

### Hitting Times by Translation Invariant Processes with Asymptotic Velocity

I am trying to prove the following statement which seems to me intuitive, but I can't work out how to formalise it. Suppose you have a stochastic process $(X(t))_{t\geq 0}$ on $\mathbb{R}^d$ which is ...
• 395
1 vote
42 views

### Existence of Fokker-Planck equation under Cauchy boundary condition.

A 1D Fokker-Planck equation within a constrain region is uniquely characterized by three functions and a boundary conditon: A drifting term $\mu(x,t)$. A diffusion term $\sigma(x,t)$. An initial ...
• 133
1 vote
29 views

• 1,711
44 views

### A question about Markov chain's definition

Background: One version of the definition of a discrete-time Markov chain is as follows: Let $S$ be a countable set. A stochastic process $(Z_n)_{n\in\mathbb{N}_0}$ is called a discrete-time Markov ...
• 33