Questions tagged [markov-process]

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.

1,537 questions
62 views

Find a modified coupling $((X_n,\tilde Y_n))_{n∈ℕ_0}$ with the same coupling time $τ$ and $\tilde Y_n=X_n$ for $n≥τ$ in the coupling lemma

Let $(\Omega,\mathcal A,\operatorname P)$ be a probability space $(E,\mathcal E)$ be a measurable space $(X_n)_{n\in\mathbb N_0}$ and $(Y_n)_{n\in\mathbb N_0}$ be independent $(E,\mathcal E)$-valued ...
30 views

If $X$ is strongly Markovian at $\tau$ is $X_{\tau+\;\cdot\;}$ a Markov process?

Let $(\Omega,\mathcal A,\operatorname P)$ be a probability space $I=[0,\infty)$ or $I=\mathbb N_0$ $(\mathcal F_t)_{t\in I}$ be a filtration on $(\Omega,\mathcal A)$ $(E,\mathcal E)$ be a measurable ...
21 views

84 views

41 views

Picking path at random in DAG graph with probability equals to path weight.

I'm refering to the following paper: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.329.3653&rep=rep1&type=pdf. There is a lemma states: Let $G$ be a directed acyclic graph with ...
63 views

time between transitions in continous time discrete state Markov process

Problem Statement: I want to compute the time between transitions in a birth-death model. As a simple example, consider that individuals are born with rate $\lambda$ and they die at rate $\sigma n$. ...
27 views

32 views

142 views

Invariant measure for Itō diffusion

Let $f\in C^2(\mathbb R)$ be positive and $h\ge 0$. Assume that $g:=f'/f$ is Lipschitz continuous and let $U$ be a strong solution of $${\rm d}U_t=\frac h2g(U_t){\rm d}t+\sqrt h{\rm d}W_t$$ ($W$ being ...
30 views

Number of returns simple random walk $\mathbb{Z}^d$

I am interested to know the precise numerical value of the expected number of returns to the origin of a simple random walk on $\mathbb{Z}^d$, when $d \geq 3$. Does anyone know where I can find such a ...
39 views

Markov Chain first order applied to attribution modelling

Based on this article I'm using within R the Channel Attribution package to leverage on the Markov Chain in order to attribute conversion between several marketing channels. However, the computation ...
40 views

58 views

10 views

Why conditional independence assumption in GP models is usually valid? Or it isn't?

I'm interested in the ground for making conditional independence assumption (e.g. that different target dimensions do not covary for given input $x$) when we are modelling some multidimensional signal ...
20 views

Markov property for mixed joint densities

Let $X \rightarrow Y \rightarrow Z$ be a Markov chain in that order, $X$ and $Y$ be jointly Gaussian, and $Z$ be a discrete random variable with finite alphabet $\mathcal{Z}$. Denote their mixed ...
136 views

83 views

Transition rate matrix of a combined birth-death processes.

If the transition rate matrix X of one birth-death process is defined by \begin{bmatrix} -\lambda & \mu \\ \mu & -\lambda \end{bmatrix} and another transition rate matrix Y of a second birth-...
21 views

How do I interpret the following $E^{X_S}[f(X_t)]$?

Given a Markov family $X=\{X_t, \mathcal{F}_t, t\ge 0\}$ on some $(\Omega,\mathcal{F})$,together with a family of probability measures $\{P^x\}_{x \in \mathbb{R}^d}$ on $(\Omega,\mathcal{F})$ and a ...
59 views

59 views