# Tagged Questions

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space ...

18 views

### Communicating classes of a power of the irreducible transition matrix? [on hold]

Suppose $P$ is an irreducible transition matrix, with period $d$. Consider the transition matrix $P^k$. In terms of $d$ and $k$, how many communicating classes does $P^k$ have, and what is the period ...
7 views

Suppose $(S_n)_{n\geq1}$ is a Markov chain on the two dimensional lattice of the integers. Then define the stopping time $\tau_A'=\inf\{n\geq1:S_n\notin A'\}$ and consider the following for $A\subset ... 0answers 26 views ### Markov Process converges intuition To explain my question, I think best to start with the example: assume Markov matrix like this:$ 0 < a < 1$$$P = \begin{bmatrix} a & (1-a) \\ (1-a) & a\end{bmatrix}$$ The question is ... 1answer 19 views ### Optimal average utility of the processing network needed In "Utility Optimal Scheduling in Processing Networks" by Michael J. Neely et al an example of processing network is provided. There are three queues ($q_1,q_2,q_3$) in the network and two processors (... 0answers 33 views ### Fractional powers of Markov generators Let$H$be the generator of a symmetric Markov semigroup on$L^2(\mathbb{R}^n).$Why the fractional power$H^\alpha$(defined on a proper domain) with$0 < \alpha < 1$turn out to be the ... 0answers 33 views ### Determining the infinitesimal generator of a Markov chain [closed] The infinitesimal generator of a Markov chain$X$on a countable state space$S$is defined by $$A(f)(x)=\lim_{t\downarrow 0} \frac{E^x(f(X_t))-f(x)}{t}.$$ Are there any ways of working out$A$... 0answers 14 views ### Coupling a “partially” stationary process? Take the stationary process$X$on$\{0,1\}$with distribution$\pi=(\pi_0,\pi_1).Then introduce the rates: \begin{aligned} 0\rightarrow2 & \quad \text{ at rate } \quad \gamma_{02} \\ 1\... 0answers 148 views ### Limit distributions for Markov chains X\to\sqrt{U+X} This question spawned from a recent, very interesting problem. Let \varphi=\frac{1+\sqrt{5}}{2} and T denote the map on the space of continuous probability density functions supported over \... 1answer 22 views ### concise book on MDPs with stress on solving them using DP What is a good book for MDP with a stress on solving them using DP? However, the book should stress on the theorems and proofs and make a case for why DP is the most popular tool to solve MDPs. I am ... 0answers 22 views ### Is the infinitesimal generator for Lie groups the same as the infinitesimal generator of a Markov semigroup? Is the infinitesimal generator for Lie groups related to the infinitesimal generator of a Markov semigroup? Or are they totally different concepts? https://en.wikipedia.org/wiki/Lie_group#... 0answers 10 views ### Filtering/MCMC methods for this HMM I have a Discrete HMM with hidden Markovian signals of the form \{X_t\}_{t \in [0, \infty)} \in \{ 1,2,3\} and observed outputs of the form \{Y_n\}_{n \in \mathbb{N}} \in \{ 1,2\}. Each ... 1answer 30 views ### Steady state distribution needed I have a chain C_t. At every instant t an exponential random variable X_t with parameter \lambda is added to the chain or if the chain has a value greater than Q then a value Q is ... 0answers 67 views ### Convergence of Markov process as some rates tend to infinity Take the simple two state Markov process characterized by transitions \begin{aligned} 0\rightarrow1 & \quad \text{ at rate } \quad \alpha\lambda \\ 1\rightarrow0 & \quad \text{ at rate } ... 0answers 92 views ### Probability of losing lotteries needed A person earnsx_i$amount of money every month where$x_i$is an exponential random variable with parameter$\lambda_1$. The amount$x_i(1-p)y$, here$0 \leq p\leq 1$and$y$is exponentially ... 0answers 14 views ### how to check if a process satisfies the markovian property with continuous time? as an example we have A source transmitting messages is alternately on and off. The off-times are independent random variables having a common exponential distribution with rate α and the on-... 1answer 77 views ### Is the mapping “positive stochastic matrix onto its Perron-projection” continuous? I am dealing with a topological question concerning the mapping that maps a positive stochastic matrix onto its invariant distribution. I am asking myself if such a mapping is continuous (or ... 0answers 12 views ### Maximum property, Resolvent, Markov process I have a question about Markov processes and related topics. Let$E$be a locally compact separable metric space and$(X_{t},P_{x})$a Markov process on$E$. For a bounded measurable function$f : E \...
Consider a Galton-Watson process, $W_0$, $W_1$, $W_2$ $\ldots$, where $W_0=1$ and the next random variables are defined by the following recursion, $$W_t = \sum\limits_{i=0}^{W_{t-1}} \xi_i,$$ where ...