# Questions tagged [markov-process]

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.

1,504 questions
14 views

161 views
+100

### If $X^{(n)},X$ are càdlàg and $X^{(n)}\to X$ in distribution, do the corresponding transition semigroups strongly converge?

Let $\left(\kappa^{(n)}_t\right)_{t\ge0}$ and $(\kappa_t)_{t\ge0}$ be Markov semigroups on $(\mathbb R,\mathcal B(\mathbb R))$ for $n\in\mathbb N$ $(T_n(t))_{t\ge0}$ and $(T(t))_{t\ge0}$ be strongly ...
20 views

### How to show that discrete-time Markov chain run backwards in time is again a Markov chain

I have a homework question on Markov Chains, can someone please give some help. The full question is: Show that a discrete-time Markov chain run backwards in time (from some time n and state i) is ...
17 views

21 views

17 views

### Partial Sums of Markov Chain.

Let $X_n$ be a independent identically distributed sequence of integer valued random variables. Suppose $S_n = \sum_{k=1}^n X_k$ with $S_0=0$, and $Z_n = \sum_{j=1}^n S_j$. Does $(Z_n)$ form a ...
17 views

### Expected extinction time for birth and death process

Suppose we have a birth and death process, with the states describing the number of lives in the population. Suppose the rate of going from state i to i+1 is $\lambda$ and the rate of going from i to ...
5 views

### Can we reparameterize most processes to be Markov decision processes?

In a finite MDP, the transition probability is schematically written as $$P(s',r' | s,a).$$ This notation reflects the assumption that the environment's evolution and the yielded reward are ...
17 views

### Calculating the distribution of the number of people in a shop

This is an extract from a question that I'm thinking about. Suppose that we have a shop and clients arrive (one at a time) according to a Poisson process with some intensity, say 4 per hour. Each ...
24 views

### Are the terms Limiting distribution and Stationary distribution properly perceived?

Consider a Markov chain $(X_n)_n$ on $S=\{1, 2\}$ with initial distribution $α$ and the transition matrix $P = \begin{bmatrix} 2/3 & 1/3 \\ 2/3 & 1/3 \\ \end{bmatrix}$ Limiting ...
13 views

17 views

34 views

### Do there exist diffusions that do not solve any SDE?

Diffusions are continuous time stochastic processess having continuous paths and satisfying the strong Markov property. I know it is possible to characterize some diffusion processes as solutions to ...
16 views

### convergent process in finite time

Let $(p_t)_{t\in\mathbb{N}}$ be an stochastic process on a countable (probability measure) space. Supose it has the Markov and the Martingale properties. It converges almost surely to a random ...
18 views

### Is there any concrete connection between a regular transition matrix and aperiodicty and irreducibility of a finite-state Markov Process?

The transition matrix T = \begin{bmatrix} 3/4 &1/4 \\ 1 &0 \end{bmatrix} is clearly a regular transition matrix but the chain itself is not aperiodic (although it is irreducible), right? (...
21 views

### Definition of Factorization of Conditional Expectation

I believe this is a very silly question or I am overlooking something fairly simple but I cannot make sense of the factorization of the conditional expectation in a very concrete application: I am ...
17 views