# Tagged Questions

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

15 views

### Preposition about the Entries of the Product of Markov Matrices.

Definition: A Markov matrix is an $n \times n$ complex matrix with the sum of the elements in every column equal to 1. My task is to prove that: If A, B are Markov matrices such that $|a_{ij}|\leq1$ ...
29 views

### Random walk on d-dimensional torus

I am reading the following paper: http://arxiv.org/pdf/1602.03849v2.pdf I will explain the general setup below. Let $x\in X=\mathbb{T}^d$, where $\mathbb{T}^d$ is the d dimensional torus. Let $\rho$ ...
9 views

### Is Markov Chain property true for correlated inputs?

I have a finite state machine (FSM). At time $k$, state is $\theta^k$ and input is $x^k$. The next state $\theta^{k+1}$ and output $y^k$ are completely determined by \begin{align} \theta^{k+1} &=...
15 views

### Markov process and Doob-Meyer decomposition

$(\Omega,\mathcal{F},(\mathcal{F}_t)_{t\geq0}),\mathbb{P})$ - a filtered probability space supporting a 1-dimensional Brownian motion $B=(B_t)_{t\geq0}$, where \mathcal{F}_t=\sigma(...
41 views

45 views

### Finding the stationary distribution

For a Markov process with state space $S= \{ 0,1,2,\dots\}$ The one step probabilities are: $p_{0,0}=q$, $p_{0,2}=p$ and $p_{i,i-1}=q$, $p_{i,i+1}=p$ for $i \geq1$ where $p+q=1$. The one step ...
18 views

### showing that a matrix has repetitive values?

Here my primary aim is to calculate the stationary distribution of a DTMC using left-eigen values i.e, $\pi = \pi*P$. But for some matrices, I observe that some states a same stationary probability. ...
26 views

### What is the main difference between Markov renewal process and Semi-Markov Process?

In The literature, it was said that Semi Markov processes are a continuous-time extension of Markov Renewal Process. We know ...
15 views

### Is this a Markov chain when dealing with minimum?

So in my probability studies I just encountered this: For a Markov chain $X_n$ we have a finite state space $\{1,2,...,k\}$ such that we can transit from one index to the next and from k only ...
29 views

44 views

### A Markov process with right continuous trajectories and left limits

Let $Nf(x) = \frac{1}{\sqrt{2\pi}}\int_\mathbb{R} e^{-\frac{|x-y|^2}{2}}f(y)dy, \;\; f \in b\mathcal{B}(\mathbb{R}), x \in \mathbb{R}.$ Let $X = (X_t, \mathcal{F}_t, \mathbb{P}^x)$ be a pure jump ...
52 views

### Population in a Galton Watson process

Consider a Galton-Watson process, $W_0$, $W_1$, $W_2$ $\ldots$, where $W_0=1$ and the next random variables are defined by the following recursion, $$W_t = \sum\limits_{i=0}^{W_{t-1}} \xi_i,$$ where ...
14 views

### Does EM can drive the complete data likelihood to decrease but likelihood increase?

I tried to apply the Expectation-Maximization(EM) algorithm on Hidden Markov Model(HMM). The data is simulated from the HMM exactly. Initialization: parameters not so far from the true one. E-step: ...
28 views

### conditions in which the repair shop process is recurrent (null\positive) or transient

here's the Story: Let $\epsilon_1.\epsilon_2,...$ be i.i.d numbers of machines for repair to the repair shop on mornings of days $1, 2,...$ . Assume that the shop is capable of repairing exactly K ...
27 views

### Let $\mathbf{X}$ be a Markov chain on a square find $p_{1,1}(n)$

Consider a square like this $$\begin{array}\\ 1 & - & 2\\ | & & |\\ 3 & - & 4 \end{array}$$ such that you can go from each state with chance $\tfrac{1}{2}$ to the ...
36 views

### How does a Markov process inherit its homogeneity to the embedded Markov chain?

A homogenous Markov process $\lbrace X(t),t\geq 0\rbrace$ is given and the embedded Markov chain $Y_0,Y_1,\ldots$ is defined as $Y_n:=X(T_n)$, where the $0=T_0<T_1<\ldots$ are the moments where ...
17 views

### Started Counts method for estimating transition probabilities of a discrete time markov chain

I would be very pleased if you could help me with a problem I'm having for my Bachelor's thesis. I'm working on some inventory forecasting methods and one of the method's I'd like to apply is a method ...
20 views

30 views

### Do Markov generators form a linear space?

Let $G_1$ and $G_2$ be generators for two distinct continuous-time Markov processes $X^{(1)}$ and $X^{(2)}$ on a common probability space $\Omega$ (with Markov semigroups $S^{(1)}$ and $S^{(2)}$) so ...
33 views

### About the expected transitions in Markov Chain

The problem is here: The given answer is here: K = $2+ X_1 + X_2$, where $X_1$ and $X_2$ are independent exponential random variables with parameters $2/3$ and $3/5$.  E[K] = 2=2+1/p_1 +1/p_2 = ...
91 views

### Crossing of Brownian Motion Sample Paths

I would like to ask for a more rigorous statement and proof of Lemma on page 5 of this paper. In essence, it states that two distinct sample paths of a Brownian motion does not strictly cross (meaning ...
### $\sum_{i=1}^{n-k}\frac{i}{n-k}\binom{2n-2k}{n-k+i}\frac{1}{2}^{2(n-k)}=\frac{1}{2}^{2n-2k}\binom{2(n-k)-1}{n-k}$
2$\sum_{i=1}^{n-k}\frac{i}{n-k}\binom{2n-2k}{n-k+i}\frac{1}{2}^{2(n-k)}=2\frac{1}{2}^{2n-2k}\binom{2(n-k)-1}{n-k}$. This is an identity in a note for a class in Markov Processes, but I can't ...