# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

68 views

### Can Continuous Time Markov Chains be used as a reasonable voting system?

I just compared a couple of example elections, as given on Wikipedia to show how Condorcet-methods differ from non-Condorcet ones, to what happens if you just interpret the underlying preference ...
539 views

39 views

100 views

### Standard deviation of a quantum walk?

The standard deviation of a classical random walk with $n$ steps is $\sqrt n$ - Standard deviation of a random walk. I have read in many places that the standard deviation of a quantum walk $n$ with a ...
71 views

### Distribution of $\max_{n \ge 0} S_n$, random walk.

Say I have a random walk that's a nearest neighbor random walk on the integers where at each step the probability of moving one step to the right is $p$ and the probability of moving one step to the ...
87 views

### Weak convergence of a sequence of stationary distributions to another stationary distribution

Let $\{X_n(t) \in \mathbb{R}^+\}$ for each $t \in (0,1)$ denote a discrete time Markov chain (with time index $n$ and parameterized by $t$). For each $t$, the Markov chain $\{X_n(t)\}$ has a unique ...
250 views

### 6-digit password - a special decoding method

Consider the situation of decoding a 6-digit password that consists of the symbols A to Z and 0 to 9, where all possible combinations are tried randomly and uniformly. Consider the following ...
80 views

118 views

### maximum renewal rate of a Markov chain

Consider a Markov chain $(X_t)$ on a state-space with a countably generated $\sigma$-algebra and assume this Markov chain allows for small sets of order one. This means there exist sets $\mathfrak{S}$ ...
281 views

### estimation of transition probabilities from aggregate data

Please, O mathematicians, help me understand the approach to the problem of estimating transition probabilities given only aggregate data in Kalbfleisch & Lawless' 1984 paper "Least-Squares ...
32 views

54 views

### This is a Markov Chain?

Consider two irreducible ergodic Markov chains with the same state space $\{0, 1, . . . , N\}$, with transition matrices $P$ and $Q$ and respective stationary distributions $\pi$ and $\rho$. We ...
52 views

### Markov Chain: Steady State Distribution.

A total of $M$ balls are divided between two urns A and B. A ball is chosen uniformly at random. If it is chosen from urn A then it is placed in urn B with probability $b$ and otherwise it is returned ...
106 views

### Norris exercise: Showing $P_0[\text{no return to}\ 0]=6/\pi^2$

Consider exercise 1.3.4 of Norris' Markov Chains. The question is as follows: Let $\{X_n\}_{n\geq 0}$ be a Markov Chain with state space $S=\{0,1,2,\dots\}$. Suppose the transition probabilities ...
44 views

### A Poisson process game

An interesting puzzle I came across: For $T>1$, observe a Poisson process of rate $1$ on the time interval $(0,T)$. Every time we observe a point, we may choose to stop. To win the game, we ...
138 views

### A(nother) variation of the coupon collector's problem

I have come across variation of the coupon collector's problem that goes like this. The coupons are of $n$ different types and in infinite number (or sampled with replacement after each draw, where "...
52 views

### $X_n = 2 Y_n + Y_{n+1}$ (non)Markov Chain

Let $Y_1,Y_2,\dots$ be iid random variables with $P(Y_n=0)=1-p,\; P(Y_n=1)=p$ where $p\in(0,1)$. Define $$X_n = 2 Y_n + Y_{n+1}$$ The question is, whether $\{X_n\}$ is a Markov chain or not. ...
215 views

### Why are inter arrival times in the continuous version of discrete-time Markov chains always exponentially distributed?

I am curious whether there exist continuous time Markov processes for which the times between jumping times (which I call inter arrival times) are not exponentially distributed, but have some other ...
57 views

### Periodicity of Markov chains under cartesian product

Suppose that you have a finite state Markov chain, with $n$ states and characterized by $p_{i,j}$ the probability of reaching state $j$ from state $i$. Consider the new Markov chain with $n^2$ states ...
143 views

### Prove or disprove: If $h$ is harmonic on $E$, then $h$ is constant on each $C_i$

For a general finite Markov chain $(X_n)_{n\in\mathbb{N}_0}$ with state space $E$ and transition matrix $P=(p_{x,y})_{x,y\in E}$, not necessarily irreducible, we define the linear space of harmonic ...
72 views

35 views

### Distribution of Markov Chain at a Stopping Time

Suppose $(X_t)_{t \geq 0}$ is a Markov chain on the state space $S$ with transition probability $p$, and that $\pi$ is a stationary distribution for $p$. If $X_0 \sim \pi$, then we know $X_t \sim \pi$ ...
58 views

### Is this transformation of a Markov process again Markovian?

Let $(X_t)_{t\in\mathbb{N}_0}$ be a stationary Markov process valued in $\mathbb{R}$ and $c\in\mathbb{R}$. Is the process $(Y_t)_{t\in\mathbb{N}_0}$ defined by $$Y_t={\bf 1}{(X_t<c)}$$ again a ...
21 views

### Markov property for the gambler's ruin problem

Let $(X_n)_{n\ge 0}$ be a simple asymmetric random walk on states $0,1,\dots,M$, where $0$ and $M$ are absorbing. Initial state is $i\neq 0,M$. Let $(X_n^*)_{n\ge 0}$ be the process $(X_n)$ ...
70 views

Let's consider a discrete time Markov chain $X_n$. Let $R_{ij} = \sum_{n=0}^\infty \mathbb{1}_{\{X_n= j | X_0 = i\}}$ be the number of visits to $j$ starting from $i$, and let $f_{ij}$ be $\text{Prob}... 0answers 53 views ### Limiting products of realizations of an integer-valued Markov chain Let$(X_m)$be a finite space discrete time irreducible and aperiodic Markov chain with stationary distribution$\pi$. The state space is a finite set of positive integers$\{x_1, x_2, \dots, x_l\}$. ... 0answers 35 views ### Irreducible and positive recurrent CTMC:$\sum_{i \in S} \pi(i) c(i) < \infty$? Suppose we have a continuous-time Markov chain$X$on the countably infinite state space$S$. The Markov chain is irreducible and all states are positive recurrent. The transition rates are given by$...
$p$ is a finite Markov chain where $p(i,j)>0$ for all $i,j$. Prove a reversible stationary distribution exists for $p$ if $p(i,j)p(j,k)p(k,i)=p(i,k)p(k,j)p(j,i)$ for all $i,j,k$ This question is ...
How can I get an expression of the probability mass function of: $$Y_i=\sum_{k=1}^i f\left(\sum_{n=1}^{k} X_n\right)$$ being $x_n, n=1,2,...$ iid random variables and \$f(\...