Tagged Questions

0
votes
1answer
58 views

Canonical Markov Process

Let $X$ be a canonical, right-continuous Markov process with values in a Polish state space $E$, equipped with Borel-$\sigma$-algebra $\mathcal{E}$ and we assume that $t\rightarrow E_{X_{t}}f(X_{s})$ ...
0
votes
1answer
47 views

Rewriting Markov process

Let $X$ be a Markov proces with state space $(E,\mathcal{E})$with initial distribution $\nu$ and transition function $P_{t}$, so $$E_{\nu}(f(X_{t+s})\mid\mathcal{F}_{s})=P_{t}f(X_{s})$$ Suppose that ...
1
vote
2answers
38 views

Diffusion process. Distribution vs transition probability.

I need confirmation on the following problem: Take a SDE of the form: \begin{equation} dX_t=a(X_t,t)dt+b(X_t,t)dW_t \end{equation} where all the conditions, such that the solution $X_t$ is defined ...
0
votes
0answers
22 views

Variability in estimations over a non-ergodic/non-regular Markov process

Imagine we have a non-ergodic/non-regular Markov Process with with $n$ states. Among these $n$ states, there are $k$ absorbing states. For each of the $n-k$ non-absorbing states, it is not possible ...
0
votes
0answers
22 views

On discrete-time stochastic attractivity of linear systems

Let $m$ be a probability measure on $Y \subseteq \mathbb{R}^p$, so that $m(Y)=1$. Consider a continuous function $f: \mathbb{R}^n \rightarrow \mathbb{R}^p$. Assume that $f(0) = 0$, and that there ...
0
votes
1answer
46 views

On discrete-time stochastic attractivity

Let $m$ be a probability measure on $Y \subseteq \mathbb{R}^p$, so that $m(Y)=1$. Consider a function $f: \mathbb{R}^n \times Y \rightarrow \mathbb{R}^n$, continuous on the first arguments, ...
3
votes
2answers
75 views

Probability of Extinction in a simple Birth and Death Process

We are asked to show that the probability of extinction $\zeta=\lim_{t\to \infty} P\left(X(t)=0\right)$ given by: $$\zeta=\begin{cases}1&\text{if }\lambda\le \mu,\\ \left(\frac \mu\lambda ...
0
votes
2answers
42 views

Specifying differential equation that describes a particular set of dynamics.

There are $S$ individuals who are susceptible to infection, and $I$ who are infectious. $S + I = N$, where $N$ is the total size of the population. Each infectious transmit the disease to a ...
4
votes
1answer
51 views

Showing a process is not markov

I keep searching but I can't find any place that gives a good method of showing a process is NOT Markov. The definition I am using is that for every $s<t$ and $g$ bounded borel there is $f$ borel ...
0
votes
2answers
96 views

Expected value of stochastic process

I have the following problem: $X_1,X_2,...$ are positive identically distributed random variables with the distribution function $F(x) :=P(X_n \leq x)$ and we assume that $F(0)<1$ for all $n$. Let ...
1
vote
0answers
47 views

Conditional distributions of (higher-order) autoregressive Markov processes

If we specify an $p$-th order autoregressive process in discrete time by its transition distribution $F_{t|t-1,\ldots,t-p}$, what can be said about lower order conditional distribution where we ...
10
votes
1answer
196 views

Motivation of Feynman-Kac formula and its relation to Kolmogorov backward/forward equations?

Kolmogorov backward/forward equations are pdes, derived for the semigroups constructed from the Markov transition kernels. Feynman-Kac formula is also a pde corresponding to a stochastic process ...
0
votes
3answers
235 views

Finding the transition probability matrix, two switches either on or off..

Each of two switches is either on or off during a day. On day n, each switch will independently be on with probability [1+number of on switches during day n-1]/4 For instance, if both switches are on ...
0
votes
0answers
35 views

Markov Model Brainteaser

An orangutan and a chimpanzee each sit at a computer typing 1 character per second. The orangutan chooses each character independently from $S$ with probability $\frac{1}{27}$. The chimpanzee follows ...
0
votes
0answers
55 views

A different Markov property definition

In Shreve's Stochastic Calculus in Finance, the Markov property is defined as Definition 2.3.6. Let $(\Omega,\mathcal F,P)$ be a probability space, let $T$ be a fixed positive number, and let ...
0
votes
0answers
86 views

What are the definitions of a diffusion process and a jump process?

I have seen following different definitions of a diffusion process and of a jump process. I was wondering how they are actually defined? Also are diffusion processes and jump processes necessarily ...
0
votes
0answers
32 views

Is HMM discriminative or generative?

Wikipedia "An HMM can be considered as the simplest dynamic Bayesian network." Here. "In probability and statistics, a generative model is a model for randomly generating observable data, ...
3
votes
1answer
127 views

Multidimensional infinitesimal generator of a jump-diffusion

Let $X=\{X_t\}_{t\geq0}$ be an $n$-dimensional Markov process, defined by the SDE $$dX_t = \mu(t, X_t) \, dt + \sigma(t,X_t) \, dB_t+\beta(t-,X_{t-}) \, dN_t,$$ where $\mu, \sigma$ and $\beta$ are ...
1
vote
1answer
66 views

Is this Markov?

Consider a process $\{X_n, n\geq 0\}$ with state space $S=\{0,1,2\}$ s.t. $$ P(X_{n+1}=j | X_n=i, X_{n-1}=i_{n-1}, \dots, X_0=i_0)=\begin{cases} P_{ij}^I \ \ \ n \ \mbox{ even},\\ P_{ij}^{II} \ \ \ ...
4
votes
1answer
58 views

Fixed point of transition kernel generates martingale

Let $P^{h}, h \geqslant 0$ be a transition kernel for some homogenous Markov process $X_t$, $\mathbb{E}|X_t|<\infty$: $$ P_{X_{t+h},X_t}(A,B) = \int\limits_{A}P^h(x,B)P_{X_t}(dx) $$ where ...
1
vote
0answers
38 views

Efficient random number generation for sojourn times in semi-Markov processes

I'm doing a self-study of semi-Markov processes and was wondering if there are efficient methods for generating random numbers for sojourn times. For example, generating a bunch of random numbers from ...
2
votes
1answer
183 views

Markov process vs. markov chain vs. random process vs. stochastic process vs. collection of random variables

I'm trying to understand each of the above terms, and I'm having a lot of trouble deciphering the difference between them (note, my mathematics training isn't very strong - so please go easy on the ...
4
votes
1answer
185 views

Covariance of Brownian-motion-like processes

We know that $\operatorname{Cov}(B_s,B_t)=\min(s,t)$ if $B_t$ is Brownian motion. What is $\operatorname{Cov}(B_{f(s)},B_{f(t)})$ for some injective $f$? How can I write $B_{f(t)}$ in an Ito ...
0
votes
0answers
24 views

How to model a continuous-time Markov process?

I'm not sure if I'm using the correct nomenclature, but I wish to model a hypothetical biological system. I have a basic knowledge of discrete-time Markov chains, so let me explain the problem as best ...
1
vote
1answer
78 views

Why is the following example of a Markov process not strong Markov

$X(t) := 0 \;\; (t \leq \tau),\;\; t - \tau\;\;(t \geq \tau)$ with $\tau$ exponentially distributed. Then X has the Markov property but not Strong Markov Property. But why ???? Can someone kindly ...
3
votes
1answer
82 views

Limit of a probability regarding a random walk

Consider a random walk on the integers starting at 0, where in each step you move either 1 or 2 meters (back or forth alike). As soon as you reach either the $N$ or $N+1$ meter mark, you stop. What is ...
1
vote
0answers
57 views

“To every Q-matrix corresponds a unique Markov process.” Proving uniqueness

"To every Q-matrix corresponds a unique Markov process." I'm trying to understand Klenke's proof of the uniqueness part of this proposition. Klenke's proof Following is an adapted version of ...
2
votes
1answer
126 views

In a continuous-time Markov process, is the waiting time between jumps a function of the current state?

Two books construct Markov processes from Q-matrices using waiting times and jump chains but differ in whether the waiting times depend on the current state. Can the two be reconciled? Klenke claims ...
1
vote
0answers
32 views

To every Markov process corresponds a Q-matrix? [duplicate]

Possible Duplicate: Logarithm of a Markov Matrix It is known that to every Q-matrix corresponds a unique Markov process. Does the converse hold? Specifically, Can a (discrete state, ...
1
vote
1answer
109 views

To every Q-matrix corresponds a unique Markov process

"To every Q-matrix $q$ corresponds a unique Markov process." I'm trying to understand Klenke's proof of the "existence" part of this proposition, namely that given a Q-Matrix $q$, there exists a ...
1
vote
2answers
91 views

A Poisson process's distributions

The Poisson process $\left(N_t\right)_{t\in\left[0,\infty\right)}$ is supposed to be a Markov process, but a Markov process $\left(X_t\right)_{t\in I}$ should be coupled with a family of distributions ...
2
votes
1answer
143 views

Showing that the Poisson process is characterized by five properties

Klenke defines the Poisson process as a family of non-negative integer valued random variables with independent increments, whose increments are distributed according to the Poisson distribution ...
0
votes
0answers
75 views

Reversibility of Markov Process and Exponential Distribution of Transition Rates

I am reading the paper Towards Utility-optimal Random Access Without Message Passing by J. Liu, Y. Yi, A. Proutiere, M. Chiang, H. V. Poor. A sentence in Section 3.2 can be paraphrased as follows: ...
0
votes
1answer
69 views

The strong Markov property with an uncountable index set

The following definition of the strong Markov property, from Klenke's book, supposes an index set $I$ that is not necessarily countable. However, it is explicitly mentioned previously (following Lemma ...
2
votes
1answer
223 views

Why does a time-homogeneous Markov process possess the Markov property?

Klenke defines (Definition 17.3, p. 346) a time-homogeneous Markov process independently, rather than as a special case of a stochastic process that possesses the Markov property (Definition 17.1, p. ...
2
votes
1answer
69 views

Markov property w.r.t. a countable state space

Background Let $\left(X_t\right)_{t \in I}$ ($I\subseteq\mathbb R$) be an $E$-valued stochastic process ($E$ being a Polish space with the Borel $\sigma$-algebra $\mathcal{B}\left(E\right)$) equipped ...
2
votes
1answer
83 views

Expected number of jumps in regular jump HMC

Consider a homogeneous Markov Chain $X$ on a countable state space, ie a jump process. It is said to be regular (does not explode) if there are only a finite number of jumps in every finite interval. ...
1
vote
1answer
67 views

Martingale Problem and PDE's

Let $X$ be a RCLL Markov Process with generator $A$. Then I know that $$ M^f = f(X)-f(X_0)-\int Af(X_s)ds $$ is a martingal for every $f\in \mathcal{D}_A$. If we suppose that $Af=0$, we see that ...
2
votes
1answer
620 views

Transition Kernel of a Markov Chain

Supposing $X_t$ is a Markov Process, can the transition kernel be defined by $$K_t(x,A):= P(X_{t+1} \in A | X_t = x)?$$ Assume that $X_t : \Omega \to \mathbb{R}^n$. The issue is that under the normal ...
0
votes
1answer
153 views

How are the pairs of two independent pure-birth processes a Markov process?

A pure-birth Process is a generalization of a homogeneous Poission process. Whereas in the Poisson process the holding times between jumps are iid exponentially distributed random variables with ...
2
votes
0answers
122 views

Is every killed Markov process still a Markov process?

Suppose we've got $X=(X(t))_{t\geq 0}$. $X$ is a strong Markov process with respect to filtration $\mathcal{F}_t$, taking values in some subset of $\mathbb{R}$. We take $\tau$ - a stopping time w.r.t ...
0
votes
1answer
198 views

Markov Process: Have you seen this notation and do you know what it means?

Ok, I've already posted this a minute ago, but my text deleted itself while I was editing it :-( So next try: Can you help me to understand the notation my professor uses to describe Markov ...
1
vote
0answers
91 views

Continuous-time Markov process - need help with flux and discrete-time equivalent

I have a continuous-time markov process and I need to calculate the following transition frequencies matrix (aka intensity matrix) transition probabilities all parameters which define permanence ...
11
votes
2answers
353 views

Difference in probability distributions from two different kernels

I wonder if the probability kernels of Markov processes on the same state space are close enough, does it also hold for the probabilities of the event that depend only on first $n$ values of the ...
2
votes
1answer
67 views

The property preserved under the perturbations of the kernel

For a measurable space $(E,\mathcal E)$ and a Markov kernel $P:E\times \mathcal E\to[0,1]$ there is a unique homogeneous Markov chain $X$. The first return time is defined as $$ \tau_A = \inf\{k\geq ...
3
votes
1answer
109 views

Levy processes with no positive jumps

Let X be a Levy process with no positive jumps and $\tau_y:=\inf\{t> 0: X_t > y\}$ then we have $$X_{\tau_y}=y\text{ on }\{\tau_y <\infty\}.$$ Could you explain that why? and does it hold ...