# Questions tagged [markov-process]

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.

1,505 questions
8k views

### What is the importance of the infinitesimal generator of Brownian motion?

I have read that the infinitesimal generator of Brownian motion is $\frac{1}{2}\small\triangle$. Unfortunately, I have no background in semigroup theory, and the expositions of semigroup theory I have ...
12k views

### Markov process vs. markov chain vs. random process vs. stochastic process vs. collection of random variables

I'm trying to understand each of the above terms, and I'm having a lot of trouble deciphering the difference between them. According to Wikipeda: A Markov chain is a memoryless, random process. A ...
24k views

### Why Markov matrices always have 1 as an eigenvalue

Also called stochastic matrix. Let $A=[a_{ij}]$ - matrix over $\mathbb{R}$ $0\le a_{ij} \le 1 \forall i,j$ $\sum_{j}a_{ij}=1 \forall i$ i.e the sum along each column of $A$ is 1. I want ...
703 views

### Hilbert's Barber Shop

Hilbert opens a barber shop with an infinite number of chairs and an infinite number of barbers. Customers arrive via a Poisson random process with an expected 1 person every 10 minutes. Upon arrival, ...
468 views

### Difference in probability distributions from two different kernels

I wonder if the probability kernels of Markov processes on the same state space are close enough, does it also hold for the probabilities of the event that depend only on first $n$ values of the ...
304 views

### Select a new value from last $N$ values; how long until the last $N$ are all the same?

Say first we have N distinct numbers in a line, like 1,2,3,...,N, in each round, we choose a ...
1k views

### Motivation of Feynman-Kac formula and its relation to Kolmogorov backward/forward equations?

Kolmogorov backward/forward equations are pdes, derived for the semigroups constructed from the Markov transition kernels. Feynman-Kac formula is also a pde corresponding to a stochastic process ...
184 views

938 views

### Show that Brownian motion on the unit circle is exponentially ergodic and has the uniform measure as its invariant distribution.

My search results keep bring up planar Brownian motion on the unit disk. However, I am specifically referring to $e^{jW_{t}} = [\cos(W_t),\sin(W_t)]^{T}$ where $W_t$ is Brownian motion. I am at a ...
275 views

This question spawned from a recent, very interesting problem. Let $\varphi=\frac{1+\sqrt{5}}{2}$ and $T$ denote the map on the space of continuous probability density functions supported over $\... 1answer 165 views ### Mean hitting time: reference request After answering [this question] (Expectation of a stopping time uniquely determined by a function) I was looking for the literature on the mean hitting/exit time for a discrete-time Markov process. ... 2answers 3k views ### Interpretation for the determinant of a stochastic matrix? Is there a probabilistic interpretation for the determinant of a stochastic matrix (i.e. an$n \times n$matrix whose columns sum to unity)? 1answer 4k views ### Markov chains: is “aperiodic + irreducible” equivalent to “regular”? I have two books on stochastic processes. In one book, it says that the limiting matrix is possible to find if the matrix is regular, that is, if for some$nP^n$has only positive values. The ... 1answer 3k views ### Why does a time-homogeneous Markov process possess the Markov property? Klenke defines (Definition 17.3, p. 346) a time-homogeneous Markov process independently, rather than as a special case of a stochastic process that possesses the Markov property (Definition 17.1, p. ... 4answers 1k views ### Probability of going into an absorbing state If I have a random walk Markov chain whose transition probability matrix is given by $$\mathbf{P} = \matrix{~ & 0 & 1 & 2 & 3 \\ 0 & 1 & 0 & 0 & 0 \\ ... 1answer 13k views ### Kendall notation's “General distribution”, what does that mean? The first and second parameters for the Kendall's notation may have a G value, which stands for General distribution, see here. But what does that mean? What is a ... 1answer 6k views ### Transition Kernel of a Markov Chain Supposing X_t is a Markov Process, can the transition kernel be defined by$$K_t(x,A):= P(X_{t+1} \in A | X_t = x)?$$Assume that X_t : \Omega \to \mathbb{R}^n. The issue is that under the normal ... 1answer 148 views ### If A is the generator of (P_t), then A+f is the generator of (P_t^f) Let X=(X_t)_{t\geq0} be a Markov process on a state space \Gamma (a Hausdorff topological vector space), let A be the infinitesimal generator of X and let \mathcal C(\Gamma) the space of ... 1answer 166 views +100 ### If X^{(n)},X are càdlàg and X^{(n)}\to X in distribution, do the corresponding transition semigroups strongly converge? Let \left(\kappa^{(n)}_t\right)_{t\ge0} and (\kappa_t)_{t\ge0} be Markov semigroups on (\mathbb R,\mathcal B(\mathbb R)) for n\in\mathbb N (T_n(t))_{t\ge0} and (T(t))_{t\ge0} be strongly ... 4answers 318 views ### Discover where Bob is sleeping using hidden Markov chains Bob lives in four different houses A, B, C and D that are connected like the following graph shows: Bob likes to sleep in any of his houses, but they are far apart so he only sleeps in a house ... 5answers 2k views ### Have any discrete-time continuous-state Markov processes been studied? I have seen discrete-time discrete-state Markov processes (such as random walks), continuous-time discrete-state Markov processes (such as Poisson processes), and continuous-time continuous-state ... 1answer 505 views ### How is the Chapman-Kolmogorov Equation not a fancy name for matrix multiplication? The Chapman-Kolmogorov Equation:$$p^{m+n}(i,j)=\sum_kp^m(i,k)p^n(k,j)$$Matrix Multiplication (with [A]_{i,j}=a_{i,j} where A is a linear map "" for B)$$[AB]_{i,j}=\sum_ka_{i,k}b_{k,j}$$In ... 3answers 560 views ### Does an n-order Markov chain still represent a Markov process? I am trying to understand Markov processes but am still confused by their definition. In particular, the Wikipedia page gives this example of a non-Markov process. The example is of pulling different ... 1answer 146 views ### Continuous-time Markov Chain question Consider an immigration-death model X = (X_t)_{t\geq0}, i.e. a model where immigrants arrive according to a Poisson process with rate \lambda and individuals have independent Exp(\mu) lifetimes. ... 1answer 280 views ### Question about Markov chain We know that if \{X_n\} is a Markov chain, then X_{n+1} is independent with the past states X_0,\ldots,X_{n-1} given current state X_n, that is$$P\{X_{n+1}=j|X_0=i_0,\ldots,X_n=i\}=P\{X_{n+1}=... 1answer 543 views ### Infinitesimal generator of Brownian motion with additional jumps A compound Poisson process is a jump process with two parameters, the rate of the jumps$\lambda$and the distribution of the jumps$\mu$($\mu$is a probability measure on$\mathbb{R}$). The ... 1answer 1k views ### Multidimensional infinitesimal generator of a jump-diffusion Let$X=\{X_t\}_{t\geq0}$be an$n$-dimensional Markov process, defined by the SDE $$dX_t = \mu(t, X_t) \, dt + \sigma(t,X_t) \, dB_t+\beta(t-,X_{t-}) \, dN_t,$$ where$\mu, \sigma$and$\beta$are ... 1answer 191 views ### Convergence of the distribution of the Langevin diffusion to its invariant measure Let$(X_t)_{t\ge0}$be a solution of $${\rm d}X_t=-h'(X_t){\rm d}t+\sqrt 2W_t,\tag1$$ where$(W_t)_{t\ge0}$is a Brownian motion and$h$is such that$X$is the unique strong solution of$(1)$. Assume ... 1answer 189 views ### If$M$maps all probability vectors on a subspace to some probability vectors, is$M$the restriction of a column stochastic matrix? For$n \ge 1$, let $$\Delta^{n-1} := \left\{ (x_1,\dots,x_{n}) \in \mathbb{R}^{n} \mid \sum_{i=1}^{n} x_i=1,~x_i \ge 0 \right\}$$ and let$\mathcal{S},~\mathcal{S}'\subset\mathbb{R}^n$be subspaces ... 1answer 135 views ### Martingale converges to the boundary I asked an almost same question before and it is solved by considering adjacent$Z_n$can not be far away and obtain a contradiction. However, if the setting is altered a bit, I wonder whether it is ... 0answers 250 views ### Poisson: Conditional Probability on Pizza order I am not sure about my answer. In particular, part b of the following question. Pizza orders arrive according to a Poisson process of rate 20 per hour. Orders are independently for a vegetarian ... 1answer 424 views ### markov chain: 2 state chain I have a machine. It has two states, broken or working. If it is working, then it will be broken with probability$q=0.1$. If the machine is working, I will make \$1000 dollar a day. If it is broken, ...
I am solving a problem in Mathematical Statistics by Jun Shao Let $\{X_n \}$ be a Markov chain. Show that if $g$ is a one-to-one Borel function, then $\{g(X_n )\}$ is also a Markov chain. Give an ...