A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

learn more… | top users | synonyms

1
vote
1answer
14 views

Escape time for a not absorbing state

Let $X$ be a right-continuous Feller Dynkin process. For $r>0$ we define the $\{\mathcal{F}_t\}_t$ stopping time (which is called escape time) $$\eta_r=\inf\{t\geq 0: \|X_t -X_0\|\geq r\}$$ We have ...
1
vote
0answers
20 views

Finding the generating function of $H_{0}$ probability of hitting 0 in Markov Chain

Let $Y1 , Y2,...$ be independent identically distributed random variables with $\mathbb{P}(Y1 =1)=\mathbb{P}(Y1 =-1)=1/2$ and set $Xo=1,Xn =Xo+Y1+...+Yn$ for $n\geq1$. Define; $$H_o= inf\{n\geq0:Xn = ...
1
vote
1answer
36 views

What is the probability of arrive either A or B at starting point K?

There are two points which are $A$ and $B$. The distance between $A$ and $B$ is $50$ meter. One person goes to $A$ with probability $\frac{1}{6}$, he goes to $B$ with probability $\frac{3}{6}$. And he ...
0
votes
0answers
26 views

Independence of Poisson processes watched only some of the time

Let $(X_t)$ and $(Y_t)$ be independent homogeneous Poisson processes with rates $\lambda,\mu > 0$, and let $t_1, t_2, \dots$ and $t_1', t_2', \dots$ be two increasing sequences of possibly infinite ...
1
vote
0answers
17 views

What is the pdf of $X$, where $dX_t = -aX_t + d N_t, N_t$ is a compound Poisson process?

I would like to find the probability density function (at stationarity) of the random variable $X_t$, where (I'm not sure this notation makes sense, I'm not very familiar with the stochastic calculus ...
0
votes
0answers
16 views

function defined as integral of borel function

I know that $f \in B_b(E)$, where $B_b(E)$ is the set of Borel bounded function on an euclidean space E. I have to show that: \begin{equation} x \to \int_{0}^{+\infty} e^{-at} P_tf(x) dt ...
0
votes
0answers
22 views

Shannon's definition of ergodicity

In A Mathematical Theory of Communication (1948) Shannon gives a definition of ergodicity for a Markov process. In order to be ergodic the directed graph of the process must have the following ...
1
vote
1answer
92 views

Stationary Markov process properties

Let $X$ be a right-continuous process with values in $(E,\mathcal{E})$, defined on $(\Omega, \mathcal{F}_t,P)$. Suppose that $X$ has stationary, independent increments. I now want to show the ...
-5
votes
0answers
43 views

Distinct states of a Markov chain [closed]

I don't understand this problem,mostly part a. Can you explain me?
-3
votes
1answer
52 views

Markov chain problem 13 [closed]

I have this problem I don't understand, Can you help me, please?
0
votes
2answers
30 views

Alternating Markov process

Given the situation: When Bob enters the room and the light is off, he turns it on with $P = 1/2$ when it is on, he does nothing. When Alice enters the room with light on, she turns it off with $P ...
0
votes
1answer
19 views

Probability equals rate $\times$ time?

Suppose a random event occurs at a rate $r$ (that's the average number of events per unit time). I have seen a number of books and papers claim that the probability $P$ of one or more events ...
1
vote
1answer
157 views

Markov process on an Abelian group

Let $E$ be an Abelian group. Let $X$ be a right-continuous process with values in $(E,\mathcal{E})$ (where $\mathcal{E}$ denotes the $\sigma$-algebra on $E$), defined on $\Omega, \mathcal{F}_t,P)$. ...
-1
votes
0answers
24 views

In Markov chains, does $(I-N)^{-1}$ always exist? [duplicate]

Spins-off from these two questions. Under what conditions does $(I-N)^{-1}$ exist? If the entries of an invertible matrix N are between -1 and 1, is its operator norm less than 1? Apparently, in ...
-1
votes
0answers
14 views

transition probability for reflected brownian motion [closed]

I have to show that the transition probability for the reflected Brownian is given by $p_+(t,x,y)=[p(t,-x,y)+p(t,x,y)]$ where $p_+=\frac{1}{\sqrt{2\pi t}}e^{-|x|^2/2t}$ and also show that it is a ...
1
vote
0answers
9 views

Markov Semigroups worked example

I have been reading this excellent paper on Markov semigroups, in which the assertion is made that a markov semigroup $\mathcal{P: L^1 \longrightarrow L^1}$ is defined by $\frac {d\mu}{dm}$ for some ...
0
votes
1answer
31 views

If the entries of an invertible matrix N are between -1 and 1, is its operator norm less than 1?

For Euclidean norm. If so, why? If not, might $(I-N)^{-1}$ exist some other way? This spins-off from here.
2
votes
2answers
64 views

Under what conditions does $(I-N)^{-1}$ exist?

Given an nxn matrix N and $I=I_n$, under what conditions does $(I-N)^{-1}$ exist? On one hand $(I-N)(I + N + N^2 + ...) = (I + N + N^2 + ...) - (N + N^2 + ...) = I?$ On the other hand, $(I-N)(I + N ...
0
votes
1answer
32 views

Distribution of particles at infinite time

Let any site of $\mathbb{Z}$ host a number of particles $\eta_0(x)$ which is distributed according to some probability distribution independently and identically for any site $x \in \mathbb{Z}$. At ...
0
votes
1answer
19 views

Weak Markov property implies strong Markov property for discrete time

From Klenke, p. 356: Theorem 17.14: If $I \subset [0,\infty)$ is countable and closed under addition, then every Markov process $(X_n)_{n\in I}$ with distributions $(\mathbf{P}_x)_{x\in E}$ has ...
1
vote
0answers
56 views

Ornstein-Uhlenbeck a Markov process

Consider the Ornstein-Uhlenbeck process defined by $$ X_t = e^{- \alpha t} X_0 + \sigma \int_0^t e^{ \alpha (s-t)} d W_s$$ with $\sigma,\alpha>0$. In many literature I have found they considered ...
0
votes
0answers
4 views

Long time statistics of random functions

I'd like to understand if an average over random functions can be approximated with a Markov process in the long-time limit. Let $$ X_t = \sum_k a_k \cos(\omega_k t + \phi_k) $$ a random function, ...
0
votes
0answers
21 views

Example of Markov process not strong Markov

This is again a question about this example (also see here). It seems we can write this process as $$X_t := \bigl(t - \text{Exp}(1) \mathbf{1}_{\{X_0 = 0\}}\bigr)^+ + \mathbf{1}_{\{X_0 \neq 0 ...
1
vote
1answer
35 views

Markov property when conditioning on future event

I am reading through an introduction to h-transform (available on https://linbaba.wordpress.com/2010/06/02/doob-h-transforms/), and came upon the following equality: $$P\left(X_{t+s}=y;X_T\in ...
0
votes
0answers
11 views

distribution of the length for a random walk on an infinite 2D grid

In connection with the flatland paradox, consider a 2D-random walk $(X_n)$ on $\mathbb{Z}^2$: the four moves of length one to W,E,N, and S are equaly likely at each time. For a fixed number of moves ...
0
votes
0answers
25 views

Generator of Wiener process and its running maximum

If we let $W$ be a standard linear Wiener process issued from zero and $M$ its running maximum $$ M_t := \sup \{ W_u: u \leq t \}, $$ then we could show that $(X,Y):=(M,M-W)$ is a Markov process on ...
2
votes
0answers
53 views

Absorbed brownian motion is a Markov process

I have been asked to prove that the Brownian motion absorbed at the origin is a Markov process. Formally, let $B_t^x$ be a Brownian motion originating from $x>0$ and let $\tau^x_0 = \inf\{t>0 : ...
0
votes
0answers
2 views

Simple Hidden Markov Model with Autoregressive Structure - Estimation?

I observe a two series over time $Y_{1:T}=\{ Y_{1}, \dots, Y_{T}\}$ and $X_{1:T}=\{ X_{1}, \dots, X_{T}\}$ where the $X$ series supposed to be exogenous (I do not define any stochastic proecess for ...
0
votes
0answers
32 views

Transition function for absorbed Brownian motion

I need an help with the following exercise. I've already seen this question Prove that Brownian Motion absorbed at the origin is Markov but I don't understand the answer. Also I would like to prove ...
1
vote
0answers
24 views

Infinitesimal generator of a semigroup

I know that if $\{T_t, t>0 \}$ is a conservative Markov semigroup on E, and $f \in D(A)$ has an absolute maximum in x then $Af(x) \le 0$. Where $D(A)$ is the infinitesimal generator of $T_t$. I ...
0
votes
0answers
54 views

Limit of a sequence of Marlov processes

Let $(X_n)$ be a sequence of Markov processes on, say, $[0,1]$, that converges in finite dimensional distributions to a process $X$. Is it true that $X$ must also be a Markov process ?
0
votes
0answers
21 views

markov process and markov chains

I have learned that Markov processes are stochastic processes possessing certain mathematical properties (memoryless, etc). My question is, if you say that a process is Markov, is it automatic (as a ...
3
votes
0answers
28 views

Martingale converges to the boundary

I asked an almost same question before and it is solved by considering adjacent $Z_n$ can not be far away and obtain a contradiction. However, if the setting is altered a bit, I wonder whether it is ...
5
votes
1answer
180 views

Convergence of Ornstein-Uhlenbeck process as a scaled Brownian Motion

Let $W$ be a standard Brownian motion. Let $\alpha,\sigma^2 >0$, and let $X_0$ be a $\mathbb{R}$-valued random variable with distibution $\nu$ that is independent of $\sigma(W_t,t\geq 0)$. Now ...
0
votes
0answers
15 views

Diagonalization of Markov Matrices

If we're given a $ \displaystyle 2 \times 2 $ Markov Matrix (so all entries are non-negative and columns add to 1) M$(a,b)$ such that $$M = M(a,b) := \begin{bmatrix}1 - a & b\\a & 1 - b ...
0
votes
0answers
9 views

Example of Semi Markov Process, that isn't a Markov Chain in Continuous Time?

Question says it all I hope. I have an exam in Stochastic Processes tomorrow and one question that may be asked is to give an example of a Semi-Markov Process that isn't a Markov Chain in Continuous ...
1
vote
1answer
36 views

Showing that a Markov jump process is a Feller-Dynkin process

Let $E$ be a countable state space with $\sigma$-algebra $2^E$ and $X_t$ a Markov jump process with transition function $$P_t(x,y) = \sum_{n=0}^\infty e^{-\lambda t}\frac{(\lambda ...
2
votes
2answers
188 views

Is the reflected Brownian Motion a Markov process

Let $W$ be a Brownian Motion (BM). The reflected BM is defined by $X=|X_0+W|$. We need to show that this process is a Markov process w.r.t. its natural filtration and we need to compute its ...
0
votes
1answer
17 views

Eigenvector / eigenvalue pairs for a Markov Matrix

If we're given a $ \displaystyle 2 \times 2 $ Markov Matrix (so all entries are non-negative and columns add to 1) M$(a,b)$ such that $$M = M(a,b) := \begin{bmatrix}1 - a & b\\a & 1 - b ...
0
votes
1answer
15 views

Finding eigenpairs for Markov Matrices

If we're given a $ \displaystyle 2 \times 2 $ Markov Matrix (so all entries are non-negative and columns add to 1) M$(a,b)$ such that $$M = M(a,b) := \begin{bmatrix}1 - a & b\\a & 1 - b ...
0
votes
1answer
38 views

Existence of steady state distribution for finite state Markov chains

Let's assume a Markov chain has 2 recurrent classes and a transient state from which we can go to either of the recurrent classes. If one of those recurrent classes is periodic, would it effect the ...
1
vote
1answer
22 views

Birth-death process: What is the distribution of reached states before reaching an absorbing state?

Intro I am working on a birth-death process. For a given choice of parameter ($n=6$, $Wa=1$, $Wb=0.95$, see below), the transition matrix is $$\left( \begin{array}{ccccccc} 1. & 0.144928 & ...
1
vote
0answers
23 views

Merging rates on a CTMC model

first time question here. I'm having a rough time trying to represent the following CTMC. Any help would be gladly appreciated. We consider a server with a infinite buffer connected to a network. ...
0
votes
0answers
13 views

Recurrent Markov chain: probability of visiting state i precisely k times in N steps

I'm studying this Markov process with transition matrix $P$, given by \begin{equation} P=\left(\begin{array}{cccc} \mu & 1-\mu & 0 & 0\\ 0 & 0 & \mu & 1-\mu\\ \mu & 1-\mu ...
0
votes
0answers
27 views

How do I compute the expected value of a function of two correlated random variables?

I'm trying to figure out how to properly compute the expected value of a function of two random variables and constants. The two random variables determine the state transitions in an MDP: The states ...
0
votes
1answer
21 views

Limiting distribution of a Markov chain?

I have the problem below. There are n identical machines. They are all operational at time 0. The lifetime of each one is an exponential random variable with rate L. There are r repairmen (1 ≤ r ≤ ...
1
vote
1answer
66 views

Is it worth playing this game of St. Petersburg paradox?

A gambler offers you the following deal. You have to keep tossing a fair coin until you get a heads, at which point you stop and collect your winnings: if it happens after n throws, the gambler will ...
0
votes
1answer
24 views

Stationary distribution of a birth and death process

I'm supposed to determine the stationary distribution, when it exists, for a birth and death process having constant parameters $\lambda_n=\lambda$ for $n=0,1,2,...$ and $\mu_n=\mu$ for $n=1,2,...$ ...
1
vote
1answer
55 views

Adjustment coefficient problem

Claims arrive at an insurance company as a Poisson process {$N(t) : t \ge 0$} at rate $\lambda > 0$ and $X_i$ is the claim size of the $ith$ claim. I assume that {$X_i, i=1,2,...$} is iid ...
1
vote
0answers
21 views

Examples of Non-Markov process with continuous time and finite set of states.

What is the best real world examples of non-Markov process with continuous time, but with finite set of states?