# Tag Info

1

In fact one defines the transition kernels: $$K(\omega, j) = \Bbb{E}(1_{X_n = j} \mid \mathcal{F}_{n-1})(\omega) \quad \Bbb{P } \,a.s.$$ This means that we have a regular conditional probability that allows us to talk about the jumps of our process. The markov property consists in saying that $K(\cdot, j)$ is $\sigma(X_{n-1})$ measurable that is ...

1

As you point out, there is no guarantee that each of the states in a Markov Chain can be visited at any particular step $n$. Another good example with lots of impossible visits is a periodic chain where any one state can only be visited on every $d$th step, where $d$ is the period. More subtle are examples on countably infinite state spaces. For example, ...

1

I am not aware of any result on the large deviation bounds you mention, but you may want to start digging in the following article (and references therein), which offers a survey of the various model that have been introduced in literature to study the performance of CSMA-like algorithms: S.-Y. Yun, Y. Yi, J. Shin, and D.Y. Eun. Optimal CSMA: A survey. In ...

1

Then after the mosquito goes to $0$, the mosquito must immediately go back to $1$ next move, i.e. the whole situation restarts. Consider the case that when the mosquito goes from $1$ to $4$ without passing $0$ as a success, and the case when the mosquito goes from $1$ to $0$ without passing $4$ as a failure. You already calculated the two probabilities as ...

1

UPDATE 2: I managed to prove rigorously that the conjecture I had formulated in the first update holds: The expressions for $\pi_j$ in both formulae are off by a factor of $t$. I also figured out the most plausible reason for this mistake. Consider the partition of the state space $\{S_0,\ldots, S_{t-1}\}$ mentioned above and let $q_{ij}\equiv p_{ij}^{(t)}$ ...

1

The idea here is to realize that eventually either all reds disappear, or all greens, whichever comes first. So the limiting distribution is extremal. Assume the subset of 1000 bacteria the predator eats every time slot is independent and uniformly distributed over all options. So, after each doubling step, each one of the 2000 bacteria has a ...

1

Yes. That's it. A Markov chain is a sequence of random variables $\{X_k\}_{k\in\{0..n\}}$ representing $n+1$ subsequent states of a system, such that for all supported values $\{i_k\}_{k\in\{0..b\}}$, and $i_c$, where $0\leq a< b< c\leq n$ we have: $$\mathsf P(X_c=i_c\mid \bigcap_{k\in\{0..b\}} X_k=i_k)= \mathsf P(X_c=i_c\mid X_b=i_b)$$ So if we ...

1

That's correct. Some comments: This notion is not attached to stochastic processes. You can use a probability vector to represent a discrete distribution in a context which involves no time dependence. In general we measure the probabilities of events, which are subsets of the sample space, rather than elements of the sample space. Traditionally we name ...

1

The chain you drew is a correct example of what you were trying to do. In the second paragraph of the question, you seem to be working with a wrong idea of periodicity. Periodicity in the context of Markov chains does not refer to periodicity of the occupation probabilities of the states as a function of time (which is, as you rightly point out, not given ...

1

Yes. A Drunkards Walk follows a similar principle to Binomial Distribution; but rather than successes (and failures) in a certain number of trials, you are counting steps forward and backwards. $\mathsf P(\eta_t=m)$ is the probability that in $t$ steps of $\pm 1$ units each, you will have moved $m$ units in total; consisting of $x$ steps forward, and $y$ ...

1

Let's model your Markov chain as follows. Let $(U_n)_{n\geq 0}$ be i.i.d. random variables uniformly distributed on the unit disk in $\mathbb{R}^2$. Set $Z_0$ take values in $\bar D$, and for $n\geq 0$ let $$Z_{n+1}=Z_n+{1\over 2} \varphi(Z_n)\, U_n,$$ where $\varphi(z)=\inf\{\|x-z\|: x\in D^c\}$. Since \begin{eqnarray*}\mathbb{E}(Z_{n+1}\mid {\cal F}_n) ...

1

You have one simple error in your matrix: the last diagonal entry should be $-4$. Everything else seems fine. Anyway, you can try to use the detailed balance equations. In particular consider the case where $k,j$ differ only by $1$, since these are actually the only possible transitions. From this you get four linear equations: 5\pi(0)=4\pi(1) \\ ...

Only top voted, non community-wiki answers of a minimum length are eligible