A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

learn more… | top users | synonyms

0
votes
0answers
16 views

Proving technique used to show an equivalence to the definition of a Markov process

Let $X=(X_t)_{t\in I}$ be Markov process with values in a Polish space $E$. I want to show, that there exists a stochastic kernel $\kappa:E\times\mathcal{B}(E)^{\otimes I}\to [0,1]$ such that ...
0
votes
1answer
36 views

Assume a die is rolled repeatedly. Find the markov matrix $P$ for the random variable of the time until the next $6$.

Assume a die is rolled repeatedly. Find the markov/transition matrix $P$ for the random variable $X_r$ = the time until the next six at time $r$. My solution was: For $i,j \geq 0$, $P$ is given ...
3
votes
2answers
53 views

Markov Chains - Strong Markov Property

I'm struck with an exercise. I tried, but the results don't seem to fit to those proposed. Exercise: Two players play the following game. The one who begins draws two cards from a deck of 40 cards ...
1
vote
0answers
29 views

Autocorrelated, discrete, bounded and symmetric random walk with no edge attraction

I need to move over a discrete set of linearly organized.. let's say "Japan steps" $S=\{0,\dots,c\}, c \in \mathbb{N}^*$. My current position is given by $d \in S$. On each time step, I need to draw ...
1
vote
0answers
12 views

Factorization of the Fokker-Planck semigroup

"In the classical theory of Markov processes, the Fokker-Planck semigroup $\{T_t:t\ge 0\}$ can be factored as $T_t=N\circ U_t \circ j$, $t\ge 0$, where $j$ is an embedding, $U_t$ is a group of ...
-1
votes
0answers
34 views

$\sup B_t$ has the same distribution as $\sup C_t$ for two brownian motions $B_t, C_t$

Let $(B_t)_{t \ge 0}$ and $(C_t)_{t \ge 0}$ be two standardized brownian motions. Now why is $\sup_{t \ge 0} B_t$ distributed same as $\sup_{t \ge 0} C_t$? This is a result we assumed as trivial ...
2
votes
1answer
35 views

Markov processes and $C_0$-semigroups

A Markov process $(X_t)_{t \geq 0}$ in continuous time on $\mathbb{R}^d$ can be described by a semigroup of Markov kernels $(p_t(x,A))_{t \geq 0}$ with $p_0(x,A) = 1_{A}(x)$ and which fulfill the ...
1
vote
0answers
25 views

Simple Markov property

I want to prove the simple Markov property but I come to a point where I do not see how to conclude. I want to prove $\mathbb{E}_\nu[Z\circ\Theta_t\mid ...
0
votes
0answers
14 views

model a system with finite users as a Markov Chain

I have to model a system M/M/2 with finite users (4 users) as a Markov Chain (and then find the probality an incoming users would enter the queue being the servers busy but that's not the problem). I ...
0
votes
0answers
5 views

Looking for resources on Harris recurrence

I'm working on a problem (in a not countable space) and it seems that I could get much further with it if I can prove that a certain Markov chain is Harris recurrent (I strongly suspect that it is). I ...
0
votes
1answer
19 views

How to understand this kind of Markov chain?

There are two unidirectionally coupled processes $X_t$ and $Y_t$. The coupling is $Y_t=g(X_{t-\delta},\dots)$. The Bayesian network of the two process is described in the following figure: Now this ...
0
votes
0answers
7 views

rigorous definition of adjoint infinitesimal generator for Markov process?

A Markov process on a $\sigma$-addive measurable space $(E,\Sigma,\mu)$ can be described by a family of operators $(P_t)_{t \geq 0} ,\, P_t: L^\infty(\mu) \to L^\infty(\mu)$. These operators define a ...
0
votes
0answers
32 views

Markov Chains and transition semigoups

I'm trying to figure out what the following statement refers to. A process $X$ is markov with transitions semigroup $(K_t)_{t\geq0}$ and initial distribution $\mu$ if and only if for all ...
0
votes
0answers
13 views

How does one estimate the order of a Markov chain empirically (given the data)?

I have a string of symbols $x_1, x_2, ...., x_n$, ($n$ very large), belonging to a finite alphabet. I know that they are a result of a Markov process, but I want to find out the order of the process. ...
3
votes
0answers
38 views

Correct steps in rewriting expectation to a probability

My knowledge of measure theory and probability spaces is limited, so please keep it relatively simple. Let $\{X(t), ~ t \ge 0\}$ be a Markov process on the countable state space $\mathbb{N}_0$ with ...
2
votes
0answers
43 views

Electrostatic capacity of two spheres with changing radii

Although I have read a lot of questions and answers here, this is my first time actually posting. Feel free to suggest needed edits. My question is the following (in a simplified setting). All this ...
1
vote
0answers
64 views

Doubt Concerning Markov Property

Given a Markovian process $(X_t )_{t\geq 0 }$, is the following property accurate? $$\mathbb E \left[ f(X_{t_1}, X_{t_2},X_{t_3}) \mid \mathcal F ^X_{t_2}\right] = \mathbb E \left[ f(X_{t_1}, ...
1
vote
1answer
21 views

Condition for the existence of a Markov process from the properties of a semigroup $\{T_t\}$

In the article Diffusion processes with continuous coefficients I (1969, Stroock and Varadhan) one finds the following arguments in pages 26-27 "$(\cdots)$ for any $ \epsilon >0, \sup_{x \in ...
0
votes
0answers
41 views

Calculating the generator of a weighted transition function

Let $(P_t)_t$ be the transition function of a Feller-Dynkin process $X$. The usual Banach space of functions that the semigroup $(P_t)_t$ is working on is $C_0(E)$, i.e. continuous functions that ...
0
votes
0answers
16 views

Ergodicity property for continuous-time Harris positive Markov process

The following theorem is Theorem 13.3.3 of Meyn and Tweedie's Markov Chains and Stochastic Stability on page 328 Theorem 13.3.3. If $\Phi$ is positive Harris and aperiodic, then for every initial ...
2
votes
1answer
68 views

Expression for the quotient between two stationary states in a Markov process

I've been thinking about this problem and I would appreciate some help. Consider a finite number of states ($n$) Markov process with transition matrix $Q_{n\times n}$ with the usual properties and ...
0
votes
0answers
25 views

Probability of hitting zero

Suppose time is discrete. $X_{t+1} = X_t + x_t$. $x_t$ is of continuous value, iid with mean zero and finite variance. Let initial condition $X_0>0$, how can I prove that the probability of $X_t$ ...
0
votes
0answers
28 views

Can solution to Rubik's cube be seen from the point of view of Markov Decision Process?

Solving Rubik's cube can be thought of as a Planning problem which has : a state space $S$ a set $G \subseteq S$ of goal states (in this case singleton) actions $A(s) \subseteq A$ applicable in ...
1
vote
1answer
55 views

Is this PMF or PDF?

I am reading a technical report on expectation-maximization (EM) algorithm (http://melodi.ee.washington.edu/people/bilmes/mypapers/em.pdf) and I am confused about something. For HMMs, it defines ...
2
votes
0answers
17 views

Discrete-Time Stochastic Calculus and Stopping Times: Resources

In my measure-theoretic probability course we covered what the professor called "discrete-time stochastic calculus". Essentially, it was a three part method for computing certain quantities such as ...
0
votes
0answers
94 views

Server farm model and non-explosiveness

Consider the following server farm model. Customers arrive at a server farm according to a Poisson process at rate $\lambda$, each requesting a machine for an amount of time that has an exponential ...
0
votes
1answer
54 views

The probability of going bankrupt

The question looks like a gambling problem. But I am not sure whether it is the same or similar to gambler's ruin problem. Assume I have wealth $W_t$. At each step $t$, I encounter a random shock ...
0
votes
0answers
20 views

Partial differential equation involving a random process (literature advice)

In articles like this one (end of page one and page two), physicists often tend to treat a random process with discrete time and countable space set as a differentiable function (whose domains are ...
1
vote
1answer
17 views

Ornstein-Uhlenbeck-Model with irregular time intervals

I have data of the form: Time t Price x(t) 0 80 21 82 24 82.3 32 81.5 ... ... The point is, that the time intervals are highly ...
4
votes
1answer
49 views

Simple random walk: What is the probability that the hitting time is exactly 2n?

I refer to the random walk $(S_n)_{n \geq 1}$ where $S_n = X_1 + \cdots + X_n$ and $X_i$ are i.i.d random variables taking values $\pm 1$ with equal probability. I want to know how to show that ...
2
votes
0answers
24 views

Limit of decreasing sequences of markov time (stopping time) is markov time?

Let $(\Omega, \mathcal{F}, \{\mathcal{F}_t\}_{t \geqslant 0}, \mathbb{P})$ be a filtered probability space and let $\tau_n \geqslant \tau_{n+1}$ be a markov time (stopping time) with respect to ...
1
vote
0answers
22 views

Discrete time Pure-Birth process with population fraction

I would like to solve difference equations for a pure-birth process, where the rate of adding new nodes depends on the fraction of population. $$x_i(t+1)-x_i(t)=\alpha ...
0
votes
0answers
18 views

What is the axiomatic definition of potential kernel?

In the enter link description here, a kernel V is a Markov potential kernel if the operator I+V is invertible and its inverse is of the form I􀀀-N with N a sub-Markov kernel.The definition of Markov ...
0
votes
1answer
38 views

Escape probabilities in a random walk.

So, a lot of theory in symmetric random walks seems to concentrate on 'hitting/stopping times' and things like that. So I started wondering... How would I go about calculating the probability of ...
1
vote
0answers
40 views

Hidden Markov Model Transition Probability

I am doing my assignment and I am asked to derive transition probability of a HMM. There are Three states. H, E and T. They initially gave me the information as follow. E is followed by an H 40% ...
2
votes
1answer
70 views

Understanding a proof of the strong Markov property of Lévy processes

I don't understand the the last sentence of a proof of the Markov property for Lévy processes given in Jochen Wengenroth's textbook "Wahrscheinlichkeitstheorie" (de Gruyter, 2008). I will appreciate ...
1
vote
1answer
63 views

Escape time for a not absorbing state

Let $X$ be a right-continuous Feller Dynkin process. For $r>0$ we define the $\{\mathcal{F}_t\}_t$ stopping time (which is called escape time) $$\eta_r=\inf\{t\geq 0: \|X_t -X_0\|\geq r\}$$ We have ...
1
vote
0answers
28 views

Finding the generating function of $H_{0}$ probability of hitting 0 in Markov Chain

Let $Y1 , Y2,...$ be independent identically distributed random variables with $\mathbb{P}(Y1 =1)=\mathbb{P}(Y1 =-1)=1/2$ and set $Xo=1,Xn =Xo+Y1+...+Yn$ for $n\geq1$. Define; $$H_o= inf\{n\geq0:Xn = ...
1
vote
1answer
39 views

What is the probability of arrive either A or B at starting point K?

There are two points which are $A$ and $B$. The distance between $A$ and $B$ is $50$ meter. One person goes to $A$ with probability $\frac{1}{6}$, he goes to $B$ with probability $\frac{3}{6}$. And he ...
0
votes
0answers
27 views

Independence of Poisson processes watched only some of the time

Let $(X_t)$ and $(Y_t)$ be independent homogeneous Poisson processes with rates $\lambda,\mu > 0$, and let $t_1, t_2, \dots$ and $t_1', t_2', \dots$ be two increasing sequences of possibly infinite ...
1
vote
0answers
18 views

What is the pdf of $X$, where $dX_t = -aX_t + d N_t, N_t$ is a compound Poisson process?

I would like to find the probability density function (at stationarity) of the random variable $X_t$, where (I'm not sure this notation makes sense, I'm not very familiar with the stochastic calculus ...
0
votes
0answers
17 views

function defined as integral of borel function

I know that $f \in B_b(E)$, where $B_b(E)$ is the set of Borel bounded function on an euclidean space E. I have to show that: \begin{equation} x \to \int_{0}^{+\infty} e^{-at} P_tf(x) dt ...
0
votes
1answer
46 views

Shannon's definition of ergodicity

In A Mathematical Theory of Communication (1948) Shannon gives a definition of ergodicity for a Markov process. In order to be ergodic the directed graph of the process must have the following ...
1
vote
1answer
109 views

Stationary Markov process properties

Let $X$ be a right-continuous process with values in $(E,\mathcal{E})$, defined on $(\Omega, \mathcal{F}_t,P)$. Suppose that $X$ has stationary, independent increments. I now want to show the ...
0
votes
2answers
33 views

Alternating Markov process

Given the situation: When Bob enters the room and the light is off, he turns it on with $P = 1/2$ when it is on, he does nothing. When Alice enters the room with light on, she turns it off with $P ...
0
votes
1answer
21 views

Probability equals rate $\times$ time?

Suppose a random event occurs at a rate $r$ (that's the average number of events per unit time). I have seen a number of books and papers claim that the probability $P$ of one or more events ...
1
vote
1answer
161 views

Markov process on an Abelian group

Let $E$ be an Abelian group. Let $X$ be a right-continuous process with values in $(E,\mathcal{E})$ (where $\mathcal{E}$ denotes the $\sigma$-algebra on $E$), defined on $\Omega, \mathcal{F}_t,P)$. ...
1
vote
0answers
14 views

Markov Semigroups worked example

I have been reading this excellent paper on Markov semigroups, in which the assertion is made that a markov semigroup $\mathcal{P: L^1 \longrightarrow L^1}$ is defined by $\frac {d\mu}{dm}$ for some ...
0
votes
1answer
36 views

If the entries of an invertible matrix N are between -1 and 1, is its operator norm less than 1?

For Euclidean norm. If so, why? If not, might $(I-N)^{-1}$ exist some other way? This spins-off from here.
2
votes
2answers
70 views

Under what conditions does $(I-N)^{-1}$ exist?

Given an nxn matrix N and $I=I_n$, under what conditions does $(I-N)^{-1}$ exist? On one hand $(I-N)(I + N + N^2 + ...) = (I + N + N^2 + ...) - (N + N^2 + ...) = I?$ On the other hand, $(I-N)(I + N ...