A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

learn more… | top users | synonyms

8
votes
0answers
201 views

Uniqueness of an infinite system of linear ODEs

How to prove that $\dot{x}=ax,\space x(0)=1$ has a unique solution if $a,x$ are infinite dimensional matrices? More specifically, let $Q$ be a bounded infinitesimal generator, i.e. ...
4
votes
0answers
107 views

A Continuous-Time Markov Process Taking All Possible Values

Let $\mathbb{N}$ be the set of positive integers. For each $n \in \mathbb{N}$, let $X^{(n)}=\{ X^{(n)}(t): t \geq 0 \}$ be a Markov chain with state-space the two point set $\{0,1\}$ and $Q$-matrix ...
4
votes
0answers
152 views

Conditional probability and integrating out part of a random walk

Suppose that I have a random walk process defined by $\alpha_{t+1}$ ~ N$(\alpha_t, \omega^2)$. Given $\alpha_t$ and $\alpha_{t+2}$, I understand why the conditional formula for ...
4
votes
0answers
255 views

estimation of transition probabilities from aggregate data

Please, O mathematicians, help me understand the approach to the problem of estimating transition probabilities given only aggregate data in Kalbfleisch & Lawless' 1984 paper "Least-Squares ...
3
votes
0answers
83 views

Proof of strong Markov Property of double sided Levy Process

Let $(X_t)_{t\in\mathbb{R}}$ be a Levy Process, i.e. $X_0 = 0$ a.s., $X$ has independent and stationary increments, and almost all paths $t\mapsto X_t(\omega)$ are right continuous with left hand ...
3
votes
0answers
68 views

A Markov Chain probability, conditioned on a random time.

My question: Upon reading theory about diffusion processes, i came across an argument which i believe simplifies to this: Say we have a Borel measurable set $A$ (if it matters you can set $A=\lbrace ...
3
votes
0answers
82 views

Moment Generating Function for Brownian motion's exit of interval.

Let $B(t)$ be a standard BM. Consider the stopping time $T = \inf\{ t > 0: |B(t)| = a\},$ the usual first exit time of the interval $(-a, a).$ We can see that $\mathbb{E} e^{tT} < \infty$ for ...
3
votes
0answers
45 views

Correct steps in rewriting expectation to a probability

My knowledge of measure theory and probability spaces is limited, so please keep it relatively simple. Let $\{X(t), ~ t \ge 0\}$ be a Markov process on the countable state space $\mathbb{N}_0$ with ...
3
votes
0answers
67 views

An irreducible Markov chain is a martingale

Let $\{X_n\}$ be an irreducible Markov chain. Does exist example of such $\{X_n\}$ which is also a martingale given that: a. $\{X_n\}$ is recurrent with finite number of states (but bigger ...
3
votes
0answers
67 views

Simple random walk conditioning on non-return

Consider a simple symmetric random walk on $\mathbb{Z}$, $(S_t)_{t \geq 0}$, with $S_0=0$. Let $k$ and $j$ be two positive integers. Let $P_{k,j}$ be the probability that the walker hits the vertex ...
3
votes
0answers
153 views

Generalization of Dobrushin's Ergodic Decomposition for continuous Markov Chains

Let $T$ be the shift transformation. Let $P$ be invariant for $T$ and also define a discrete state space Markov Chain. Let $C_{1},\ldots,C_{n}$ be the connected components of the Markov Chain. It ...
3
votes
0answers
313 views

Is every killed Markov process still a Markov process?

Suppose we've got $X=(X(t))_{t\geq 0}$. $X$ is a strong Markov process with respect to filtration $\mathcal{F}_t$, taking values in some subset of $\mathbb{R}$. We take $\tau$ - a stopping time w.r.t ...
2
votes
0answers
23 views

Moving Average of an Ergodic Markov processes.

Let $ \{X(t); t\geq 0\} $ be an ergodic Markov process, and let G be a positive integrable function with $\int_0^\infty G(x)dx=1$. Does $$F(t)\doteq G*X(t) = \int_0^t G(t-s)X(s)ds $$ converge in ...
2
votes
0answers
24 views

Good introductory book coupling methods

I am very interested in coupling methods, can you recommend me a good introductory books on this subject? Thanks
2
votes
0answers
37 views

Markov chain limit problem

Let $X_n$ be a Markov chain on a countable state space, $\mathbb{S}$. Let $N_n(x) = \sum_{k=1}^n\mathbb{1}_{\{X_k=x\}}$ denote the number of times the chain visits state $x\in \mathbb{S}$. Let ...
2
votes
0answers
34 views

True or false: The process $\{ X(t), t \geq 0 \}$ at a $M/M/s$ queue is a reversible Markov process.

Let $X(t)$ denote the number of customers in a system at time $t$. The process $\{ X(t), t \geq 0 \}$ at a $M/M/s$ queue is a reversible Markov process. Is this statement true or false for: (a) ...
2
votes
0answers
54 views

Markov process and filtration

I would like to restate the question. I'm reading Revuz/Yor's definition of Markov process (P81), they started from transition function, and define the $P_t f(x)$ as usual (let's only consider the ...
2
votes
0answers
29 views

Marginalized non Markov Chapman kolmogorov equation

The usual Chapman kolmogorov equation states $$\int dx_1 p(x_2|x_1)p(x_1|x_0)=p(x_2|x_0)$$ Which means we can also identify $$\int dx_0 p(x_2|x_1)p(x_1|x_0)p(x_0)=p(x_2|x_1)p(x_1)$$ Now, because ...
2
votes
0answers
20 views

Why do we look only for the first and second derivative when dealing with diffusions?

I would like to understand, from an analytical point of view, why is it that we only take the first and second derivatives into account when computing the generator of a diffusion. This question is ...
2
votes
0answers
43 views

Excursion of random walk conditioning on return

Consider a simple random walk in one dimension starting from the origin. Let $\epsilon>0$. How to prove that, conditioning on the event that the random walk is at the origin at time $n$, the ...
2
votes
0answers
39 views

Compute the value of $\mathbb{P}_{0}\left[\tau\geq n\right]$ and $\mathbb{P}_{0}\left[\tau=+\infty\right]$

We consider the Markov chain $\left(X_{n}\right)_{n\geq0}$ on $\mathbb{N}$ whose transition matrix $P=\left(p_{k,l}\right)_{k,l\geq0}$ given by $$\forall k\in\mathbb{N} ...
2
votes
0answers
36 views

Expected response time of Continuous time Markov chain

I'm studying CTMC (Continuous Time Markov Chains). I came across the following slide I don't understand how they got $M(t+h) = M(t) + \alpha h + M(t)\lambda h - M(t) \mu h +o(h)$ Could anyone ...
2
votes
0answers
52 views

Why is the stationary distribution a distribution?

Suppose we have a time-homogeneous, discrete-time, aperiodic, positive recurrent, irreducible Markov chain $(X_t)_{t \geq 0}$ on a discrete state space $E$. It is known that its stationary ...
2
votes
0answers
105 views

How to prove a set is a core for an infinitesimal generator

I am trying to prove that the set $D=\bigcap_{n\geq 1} \{f\in C^2 (\mathbb{R}) :f(0)=\sum_{k\geq 0} f(\frac{k}{\sqrt{n}})g_n (k)\}$ is a core for the infinitesimal generator of reflecting brownian ...
2
votes
0answers
54 views

Semigroup associated to a Markov process

I'm studying the transition semigroup associated to a Markov Process, in particular the Hille-Yosida theorem and the Martingale Problem. In my notes I found : "If $\{T_t\}_t$ is a strongly continuous ...
2
votes
0answers
46 views

Simple Markov property

I want to prove the simple Markov property but I come to a point where I do not see how to conclude. I want to prove $\mathbb{E}_\nu[Z\circ\Theta_t\mid ...
2
votes
0answers
68 views

Electrostatic capacity of two spheres with changing radii

Although I have read a lot of questions and answers here, this is my first time actually posting. Feel free to suggest needed edits. My question is the following (in a simplified setting). All this ...
2
votes
0answers
26 views

Discrete-Time Stochastic Calculus and Stopping Times: Resources

In my measure-theoretic probability course we covered what the professor called "discrete-time stochastic calculus". Essentially, it was a three part method for computing certain quantities such as ...
2
votes
0answers
40 views

Limit of decreasing sequences of markov time (stopping time) is markov time?

Let $(\Omega, \mathcal{F}, \{\mathcal{F}_t\}_{t \geqslant 0}, \mathbb{P})$ be a filtered probability space and let $\tau_n \geqslant \tau_{n+1}$ be a markov time (stopping time) with respect to ...
2
votes
0answers
79 views

Absorbed brownian motion is a Markov process

I have been asked to prove that the Brownian motion absorbed at the origin is a Markov process. Formally, let $B_t^x$ be a Brownian motion originating from $x>0$ and let $\tau^x_0 = \inf\{t>0 : ...
2
votes
0answers
62 views

Transition function for absorbed Brownian motion

I need an help with the following exercise. I've already seen this question Prove that Brownian Motion absorbed at the origin is Markov but I don't understand the answer. Also I would like to prove ...
2
votes
0answers
136 views

Why are inter arrival times in the continuous version of discrete-time Markov chains always exponentially distributed?

I am curious whether there exist continuous time Markov processes for which the times between jumping times (which I call inter arrival times) are not exponentially distributed, but have some other ...
2
votes
0answers
27 views

Showing which classes are recurrent and which are transient

If I have a Markov chain on states {0,1,2,3,4,5} $$ \mathbf{a} = \matrix{~ & 0 & 1 & 2 & 3 & 4 & 5 \\ 0 & 1/3 & 0 & 2/3 & 0 & 0 & 0 \\ ...
2
votes
0answers
37 views

Define Markov chain and rewrite to recursively solve

Customers arrive at a server with rate $\lambda$ and are served at rate $\mu$. The server breaks down with rate $\gamma$, which causes all customers to leave. New customers can only arrive once the ...
2
votes
0answers
105 views

Model as a continuous time Markov Chain

A system consists of two machines, of which one works and the other is standby. Only the working machine can break down (with rate $\lambda$). If it breaks down the other machine takes over (if it ...
2
votes
0answers
63 views

Discrete Laplacian

I have the following question and I can't figure out how to do the proof. Could you give me some hints in both directions of the equivalence? Suppose $A$ is a bounded subset of $\mathbb{Z}^d$. Then ...
2
votes
0answers
27 views

Probability of going from a set $S$ to its complement on a Markov chain

I need to show that if $\pi$ is the stationary distribution of a Markov chain $M$, then for every set of vertices $S$, the probability to choose a random node in $S$ according to $\pi$ and then going ...
2
votes
0answers
69 views

Stochastic process using Markov chain (thief on the run!!)

I'm given an exercise where we are to simulate a thief escaping from an officer. The thief (let's call him/her T for simplicity) and an officer (O) have four cities to be in. Let's call the cities A, ...
2
votes
0answers
134 views

Markov Chain with Normal Transition Matrix

Consider a (sub)-stochastic matrix $P$, and the associated Markov chain $X$ with \begin{align*} \mathbf P [X_n =y | X_0 = x] = P_{xy}^n. \end{align*} Suppose we have the condition $P^T P = P P^T$, ...
2
votes
0answers
72 views

A theorem in the paper “Noncommuting Random Products” by Furstenberg

I have a question concerning the proof of theorem 2.5 at page 395 of the paper Noncommuting Random Products, by H. Furstenberg, Trans. Amer. Math. Soc., 1963. The statement is as follows: Let $\mu$ ...
2
votes
0answers
70 views

Examples of decreasing-in-some-time-interval variance of a time homogeneous Markovian process

Let $x_t$ be a zero mean, time homogeneous Markovian process over time $t$ starting from $x_0=0$. What are the examples of $x_t$ where the variance at $t$ decrease over some interval of $t$? The ...
2
votes
0answers
42 views

Exsistence and uniqueness of stationary density for Markov Chain

Suppose we're given a function $f:\mathbb{R}^2\to\mathbb{R}$. We define a Markov Chain $(X_n)$ by \begin{align} X_0&\sim f_X, \\ X_n&=f(X_{n-1},Y_{n-1}), \end{align} where $(Y_n)$ is a ...
2
votes
0answers
118 views

Interpretation of potential kernels for Markov processes

One can associate a strongly continuous contraction semi group (SCCSG) to a Markov process with state space $S$ through its transition function, say $P_t$. Now one can interpret $P_t$ as a linear ...
2
votes
0answers
101 views

References for basics of Piecewise-Deterministic Markov Processes

I am looking for introductory/pedagogical material to Piecewise-Deterministic Markov Processes (see http://en.wikipedia.org/wiki/Piecewise-deterministic_Markov_process) (For the moment I am interested ...
2
votes
0answers
93 views

“To every Q-matrix corresponds a unique Markov process.” Proving uniqueness

"To every Q-matrix corresponds a unique Markov process." I'm trying to understand Klenke's proof of the uniqueness part of this proposition. Klenke's proof Following is an adapted version of ...
1
vote
0answers
15 views

non-ordered chapman kolmogorov equation

If $p(x)$ is a probability density one would normally have $\int dx_1 p(x_2|x_1)p(x_1)=p(x_2)$ However is there a straightforward way to interpret the following: $\int dx_1 ...
1
vote
0answers
26 views

An example of Markov chain with no closed class?

What is an example of Markov chain with no closed communicating class? Closed class means that once we are in that class, there would be no escape from it. I am thinking that an example would be ...
1
vote
0answers
15 views

Applying Markov Decision Processes to an arrival forecasting problem

I have the following problem and I'd like to know if it's something that was already studied in the literature or not. I'm not sure about the naming conventions either. I have a system $S$ that can ...
1
vote
0answers
54 views

Does a Markov process have memory zero?

I have the following question: The Markov process with two states and a transition matrix $$P =\begin{pmatrix} 0.3 & 0.7 \\ 0.3 & 0.7 \end{pmatrix}$$ has memory zero. Is it true? My ...
1
vote
0answers
40 views

Birth and death process. Total time spent in state i.

Question: Let $X(t)$ be a birth-death process with $\lambda_n = \lambda > 0$ and $\mu_n = \mu > 0,$ where $\lambda > \mu$ and $X(0) = 0$. Show that the total time $T_i$ spent in state $i$ is ...