Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

17 views

Is there a way to “normalise” probability values of Markov chain transitions for comparison?

Suppose I have a series of states and I've a database of frequencies (probabilities) of the states transiting from one to another. I've a set of states ${A...G}$. Let's say I've a state transition (...
18 views

Stationary distribution of finite-state Markov chain in terms of determinants/products of eigenvalues

I have an $M$-state continuous-time Markov chain with transition-rate matrix $K$ (the column sums are zero), which has $M$ distinct eigenvalues $\lambda_i$, $i=1,\dots,M$. $\lambda_M=0$, so $K$ has ...
32 views

Equilibrium distribution exponentially fast

I need to prove that for an aperiodic, irreducible Markov Chain $X_n$ with stationary distribution $\pi$ holds that $P_x[X_n=j]\to\pi(j)$ exponentially fast. I found some proof of that statement but ...
93 views

Probability of losing lotteries needed

A person earns $x_i$ amount of money every month where $x_i$ is an exponential random variable with parameter $\lambda_1$. The amount $x_i(1-p)y$, here $0 \leq p\leq 1$ and $y$ is exponentially ...
40 views

Bayesian Estimator and Markov Chains

This is Exercise 6.1.14 from Dembo's notes found here. At this point, we are just beginning a discussion of Markov chains. I have no prior experience with estimators and so I am a bit lost with this ...
28 views

Check that stopping time is a.s. finite

I have the following situation. Let $(X_i)_{i\geq1}$ be a sequence of iid random variables in $\mathbb{Z}$ and consider the random walk $S_n=\sum_{i=1}^n{X_i}$, $S_0=x$. Let $y>x$ and consider ...
52 views

26 views

HMM limiting distribution

Consider a hidden markov model (HMM) with two hidden states $A$ and $B$ and emission support $1$ and $2$ fitted with initial state distribution $$\lambda = [\begin{array}{cc} .7&.3\end{array}]$$ ...
28 views

Long-run fraction Markov Chains

A machine has three critical parts (1,2,3) but can function as long as two of these parts are functional. When two are broken, they are replaced and the machine is functional the next day. The state ...
38 views

17 views

40 views

Probability that two random walks on $\mathbb{Z}^2$ meet at the origin

Suppose $X,Y$ are symmetric, independent random walks on the lattice $\mathbb{Z}^2$. I am trying to find the probability: $$\mathbb{P}\big(X_n=Y_n=(0,0)\;\text{for some}\;n\,\big|\,X_0=Y_0=(0,N)\big)$$...
Suppose the Markov chain with Probability Transition Matrix, $P$ = ($p{_x}{_y}$) is ergodic and $p{_m}(x, y) > 0$ for all states $x$ and $y$. If $n ≥ m$, show that $p_n(x, y) > 0$ for all ...