A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

learn more… | top users | synonyms

2
votes
2answers
71 views

Under what conditions does $(I-N)^{-1}$ exist?

Given an nxn matrix N and $I=I_n$, under what conditions does $(I-N)^{-1}$ exist? On one hand $(I-N)(I + N + N^2 + ...) = (I + N + N^2 + ...) - (N + N^2 + ...) = I?$ On the other hand, $(I-N)(I + N ...
-2
votes
2answers
417 views

Is first order moving average a Markov process?

Given first order moving average $$ x(n) = e(n) + ce(n-1) $$ where $e(n)$ is a sequence of Gaussian random variables with zero mean and unit variance which are independent of each other, and $c$ is ...
1
vote
1answer
65 views

Symmetric random walk and the distribution of the visits of some state

I need help with this problem: Let $(S_n)_n$ a symmetric random walk in $\mathbb{Z}$ i.e $S_n=X_1 + \cdots + X_n$ with $(X_n)$ iid $\mathbb{P}(X_n=1)=\mathbb{P}(X_n=-1)=\frac{1}{2}$. Let $m \in ...
2
votes
1answer
125 views

A question about a stochastic process being Markov

Let $(X_{s},\mathcal{F}_{s})$ be a stochastic process adapted to a given filtration. I was told that, in order to prove that $X$ is Markov, it suffice to prove that for any nonnegative, ...
4
votes
1answer
2k views

Why does a time-homogeneous Markov process possess the Markov property?

Klenke defines (Definition 17.3, p. 346) a time-homogeneous Markov process independently, rather than as a special case of a stochastic process that possesses the Markov property (Definition 17.1, p. ...
2
votes
1answer
86 views

Is $(B_t^2)$ Markov where $(B_t)$ is Brownian motion?

I am pretty sure $(B_{t}^{2})$ not Markov because the squared random walk is not. Showing the square of a Markov process is or isn't Markov I guess I can repeat the method since to be Markov it ...
0
votes
1answer
36 views

If the entries of an invertible matrix N are between -1 and 1, is its operator norm less than 1?

For Euclidean norm. If so, why? If not, might $(I-N)^{-1}$ exist some other way? This spins-off from here.
0
votes
1answer
81 views

probability, random walk, Markov chain question

Let $P$ be a transition matrix for a regular Markov chain and let $w$ be it’s equilibrium vector. Show that $w$ has no zero entries.
0
votes
1answer
83 views

Including non-markovian processes in a birth-death process

Current model I want to model a stochastic system with a birth-death (Markovian) model. I therefore have this kind of $n$ times $n$ (where $n$ is the number of possible states) transition matrix: ...
0
votes
1answer
89 views

Showing the square of a Markov process is or isn't Markov

Hi I am trying to show that if $X_n$ is a markov process, whether or not $X_n^2$ is a markov process. $X_n$ is a markov process if $P\{X_k = a_k|X_{k-1} = a_{k-1}, X_{k-2} = a_{k-2}, ..., X_k = a_1 ...
14
votes
2answers
1k views

What is the importance of the infinitesimal generator of Brownian motion?

I have read that the infinitesimal generator of Brownian motion is $\frac{1}{2}\small\triangle$. Unfortunately, I have no background in semigroup theory, and the expositions of semigroup theory I have ...
3
votes
1answer
280 views

Prove that Brownian Motion absorbed at the origin is Markov

I'm trying to prove that Brownian motion absorbed at the origin is a Markov process with respect to the original filtration $\{\mathcal{F}_{t}\}$. To be more specific, let $(B_{t},\mathcal{F}_{t})_{t ...
2
votes
1answer
141 views

The strong Markov property with an uncountable index set

The following definition of the strong Markov property, from Klenke's book, supposes an index set $I$ that is not necessarily countable. However, it is explicitly mentioned previously (following Lemma ...
1
vote
3answers
672 views

Estimating the transition matrix given the stationary distribution

Let's say we are given a Markov chain for variable $X = [x_1, ..., x_n]$; also we are given a desired stationary distribution for this graph $P_\infty = [p_1, ..., p_n]^\top$. How can we design an ...
4
votes
1answer
108 views

Find the Stationary Distribution of an infinite state Markov chain

A Markov Chain on states 0,1,..... has transition probabilities $P_{ij}=1/(i+2)$ for j=0,1,....,i,i+1. I'm supposed to find the stationary distribution. So do I take the limit as n goes to ...
3
votes
1answer
362 views

Question on the proof of the Blumenthal 0-1 law in textbook.

I'm studying Stochastic Processes by Richard F. Bass. Within this book I encountered the definition of a Markov process, which is given as follows: We are given a separable metric space $S$ endowed ...
2
votes
0answers
49 views

Transition function for absorbed Brownian motion

I need an help with the following exercise. I've already seen this question Prove that Brownian Motion absorbed at the origin is Markov but I don't understand the answer. Also I would like to prove ...
2
votes
0answers
87 views

“To every Q-matrix corresponds a unique Markov process.” Proving uniqueness

"To every Q-matrix corresponds a unique Markov process." I'm trying to understand Klenke's proof of the uniqueness part of this proposition. Klenke's proof Following is an adapted version of ...
1
vote
0answers
49 views

Semigroup associated to a Markov process

I'm studying the transition semigroup associated to a Markov Process, in particular the Hille-Yosida theorem and the Martingale Problem. In my notes I found : "If $\{T_t\}_t$ is a strongly continuous ...
1
vote
1answer
122 views

Stationary Markov process properties

Let $X$ be a right-continuous process with values in $(E,\mathcal{E})$, defined on $(\Omega, \mathcal{F}_t,P)$. Suppose that $X$ has stationary, independent increments. I now want to show the ...
1
vote
1answer
126 views

Strong Markov property of Bessel processes

I am thinking about the following: If $(B_t)_{t \geq 0}$ is a Brownian motion in $\mathbb{R}^3$, how can we show that the Bessel process (of order $3$) $(|B_t|)_{t \geq 0}$ has the strong Markov ...
1
vote
0answers
62 views

Follow-up on solution to Markov process equation

I asked a question here about solving a system related to an absorbing Markov chain. I now have a variation where there are $m$ types (of student, job seeker, etc) each of which applies to ...
1
vote
0answers
27 views

showing a condition implies convergence to invariant distribution

For an arbitrary Markov chain, I'm trying to show that $\lVert{ P_{\mu}\circ \theta_n^{-1} - \nu \rVert}\to 0$ iff $\sum_j \lvert{ P_{ij}^n - \nu(j)\rvert}\to 0$, where $\theta_n^{-1}$ is a shift ...
1
vote
2answers
277 views

Expected number of steps between states in a Markov Chain

Suppose I am given a state space $S=\{0,1,2,3\}$ with transition probability matrix $\mathbf{P}= \begin{bmatrix} \frac{2}{3} & \frac{1}{3} & 0 & 0 \\[0.3em] \frac{2}{3} ...
1
vote
1answer
53 views

makov chains and property

let $X_t$ for $t \in \{1,2,...,10\}$ be a sequence of independent tosses of a fair coin. We denote heads with 1 and tails with 0. Define the random variable $S_n=\sum_{t=1}^n X_t$ for $n \in ...
1
vote
1answer
158 views

Why is the following example of a Markov process not strong Markov

$X(t) := 0 \;\; (t \leq \tau),\;\; t - \tau\;\;(t \geq \tau)$ with $\tau$ exponentially distributed. Then X has the Markov property but not Strong Markov Property. But why ???? Can someone kindly ...
0
votes
1answer
182 views

Covariance of states of a finite Markov chain

I know it is possible to construct a covariance matrix for states of a Markov chain but I cannot seem to find a proper way to compute it. I will attach some theories I found from Kemeny and Snell's ...
0
votes
1answer
63 views

Is there a solution to this system for the diagonal matrix?

I'm trying to find a solution to a system of equations, but its quite different from anything I've come across before. I believe there is a solution, but I could be wrong. $\mathbf{A} = ...
0
votes
3answers
127 views

How fast does this Markov chain converge?

Observe the above a Markov chain and limiting matrix of it. Finding the limiting matrix if it exists is easy but I am curious as to how fast this given matrix converges to its limiting matrix. Is ...