Hot answers tagged

4

First, the walk needs to get from the leaf to the root. This is a simple random walk in the vertical direction, with probability $\frac23$ to go down and $\frac13$ to go up. The expected time $a_k$ to reach the root from depth $k$ satisfies $$ t_k=1+\frac23t_{k+1}+\frac13t_{k-1} $$ for $k=1,\ldots,h-2$ with boundary conditions $t_0=0$ and ...


3

The "main theorem" of this subject says that an aperiodic irreducible Markov chain on a finite state space converges to a unique stationary distribution regardless of the initial distribution. (To my knowledge this theorem has no name, but it is closely related to the Perron-Frobenius theorem from linear algebra.) If you drop the aperiodic assumption, then ...


3

The simplest one: transition matrix $\pmatrix{0 & 1\cr 0 & 1\cr}$.


3

Basically no, because this is sensitive to the distribution of $X_1$, which must be given initially. However, typically you could do it if you instead had $X_N,X_{N+1},\dots,X_{N+M}$, where $N$ and $M$ are both large. Then under generic (but not universal) circumstances, there is a unique invariant distribution, the chain converges to this distribution, and ...


2

Think about this: $\tau_1=G_1$, where $G_1$ is Geometric with parameter $\frac 12$. Continue inductively: $$\tau_{n+1} = \tau_n + G_{n+1},$$ where $G_{n+1}$ is Geometric with parameter $\frac 12$, independent of $\tau_n$. Therefore for each $n\ge 1$, $\tau_n$ is the sum of $n$ independent Geometric with parameter $\frac 12$. In particular, since $E ...


2

Elaborating on @Did's comment (letting $a_0(s)=\lambda s$ and $b_0(s)=1-\lambda$ as according to the initial distribution), we have the recurrences \begin{align} a_{n+1}(s) &= p_asa_n(s) + (1-p_b)sb_n(s),\\ b_{n+1}(s) &= (1-p_a)a_n(s) + p_b b_n(s). \end{align} Adding these yields $$a_{n+1}(s) + b_{n+1}(s) = (1-p_a+p_as)a_n(s) + (1-p_b+p_bs)b_n(s), ...


2

It's just linearity of expectation – the expected number of visits to $y$ before returning to $z$ is the sum over all times of the expected numbers of such visits at that time, which are the probabilities of the state being at $y$ at that time and not yet having reached $z$.


2

It is the first visit. $$f_{ij}^{(n)} = P(X_n = j,\: X_k \not = j, \: k=1,2,\ldots,n-1 \mid X_0 = i)$$ You can make two first revisits to $1$ by $n=2: \:\:1 \rightarrow 3 \rightarrow 1$ $n=3: \:\:1 \rightarrow 3 \rightarrow 2 \rightarrow 1$ $$f_{ij}^{(n)} = \sum_{k \not = j} p_{ik} f_{kj}^{(n-1)} $$


2

The number of offspring of a cell is $X$. The probability that $X=k$ is given by: $~\mathsf P(X=k) = qp^k$ for $k\in\{0, 1, 2,\ldots \}$ and $q=(1-p)$. The expected value of $X$ is given by : $~\mathsf E(X) = \sum_{k=0}^\infty k~\mathsf P(X=k)$ Thence, using Geometric Series closed form (when $\lvert p\rvert<1)$: $$\begin{align}\mathsf E(X) ~=~& ...


1

Let $P\in Mat_{n \times n}$ be an irreducible transition matrix. Since the rows of $P-I$ sum to $0$ (the rows of $P$ sum to $1$ and the rows of the identity matrix sum to $1$), the $rk(P-I)\leq n-1$. This implies that $\exists$ an eigenvector $v$ such that $vP=v$. Continuing with the proof, we find that if $P$ is irreducible, then $\exists$ a unique ...


1

The statement is correct with $X_\tau$, but you have to read it carefully. The LHS is a conditional expectation, hence a random variable, so for some $\omega \in \Omega$ you have that $\mathbb{E}(f(X_{\tau+1}, \ldots) \mid \mathcal{F}_\tau) = \mathbb{E}_{X_{\tau}(\omega)}(f(X_0, \ldots))$, i.e. you can restart your Markov chain at some random point and ...


1

By limiting distribution, they mean the distribution to which the Gibbs distribution converges as you take the limit $T\to0$ or $T\to\infty$. The limit $T\to\infty$ is a bit easier to take, since in this case $\beta\to0$, so you can calculate the limiting distribution directly by substituting $\beta=0$ into the expression you wrote; you don't need any ...


1

$P(\text{hot})$ isn't one. All you know is that the event "hot" occurred; this doesn't imply that it has probability 1. You need to calculate using Bayes' rule. Using $H$ to denote the event "hot", we see: $$P(A\mid H)={P(H\mid A)P(A)\over P(H)}={P(H\mid A)P(A)\over P(H\mid A)P(A) + P(H|B)P(B)}.$$ Now plug in $P(A)=0.4$, $P(B)=0.6$, $P(H\mid A)=0.9$, ...


1

The markov property specifies that the probability of a state depends only on the probability of the previous state. You can "build more memory" into the states by using a higher order Markov model. In an $n$th order Markov model $$P(x_i | x_{i-1}, x_{i-2},..., x_1) = P(x_i | x_{i-1},..., x_{i-n} ) $$ Example of a second order MC


1

There is nothing radically different about second order Markov chains: if $P(x_i|x_{i-1},..,x_1)=P(x_i|x_{i-1},..,x_{i-n})$ is a "n-th order Markov chain", we can still interpret it as a first order Markov chain, on the space of combinations of $n$ states, i.e. $S^n$, if $S$ is the set of values $x_i$ takes: just write ...


1

A common way to find $P^n$ is to diagonalize your matrix. Then you will have $P=MDM^{-1}$ with D a diagonal matrix, so $P^n=MD^nM^{-1}$. So taking $n \rightarrow \infty$ will be easy. Also, if $\mu$ is a measure of probability on your two states MC, $\underset{n\to \infty}\lim\mu P^n$, if it converges, is an invariant probability.


1

If some $y\in A$ is not recurrent, then eventually it jumps to $\mathcal S-A$ (in finite time), which eventually jumps again to $A$ in view of the full probability hypothesis. Actually, the sequence keeps doing this, and it will return infinitely often to $A$ (since it is transient, by hypothesis). But since $A$ is finite, in this sequence some element $z\in ...


1

HINT probability to never return to 0 starting at 0 is to always move to the right, i.e. $$\lim_{n \to \infty} (1/2)^n = 0$$


1

It looks like you're overcounting... The sequence $(1, 1, \dots \text{ <<996 more "1"s>> } \dots, 1, 1, 0)$ is included in your $k=1$ count, even though that walk stopped when $k=0$. You actually require that the last $X_i$ is $1$, so you only need to scatter $999$ $1$s in the preceding sequence.


1

I don't know if these will help you or not, but it looks like you might want a bivariate distribution for your emission probabilities (one variable being the folding energy and the other parameter being the codons). Here are some references I found quickly regarding the construction of a joint distribution for which one variable is continuous and the other ...


1

The sample space should, typically, contain all possible outcomes of an experiment. Presumably, the experiment here is playing a single game. So, the sample space should be a description of a game that contains all the information you find interesting. For example, if all you care about is the length of a game, then your sample space could be all natural ...


1

$M/M/1$ and its relatives are the queues which are markov chains. This is since these queues have exponentially distributed interarrival times and service times. $M/D/1$ is not a markov chain but there exists an imbedded discrete time markov chain whose properties provide information about the process. $M/G/1, G/G/1$ etc are not markov chains. But if ...


1

Just take any non-irreducible Markov chain with two communicating classes, both of which satisfy the hypotheses of the ergodic theorem. Then the limit distribution will depend on which communicating class one starts in (even though any convex combination of those two limiting distributions satisfies the requirement of stationarity). EDIT: basically look at ...


1

This isn't an answer per se, rather some thoughts to consider. If $\{X_n:n=1,2,\ldots\}$ is a Markov chain and we define the holding times $\{T_n:n=1,2,\ldots\}$ by $T_n\sim\mathsf{Exp}(\lambda_{X_n})$, then the sequence $\{T_n\}$ only defines a Poisson process if $\lambda_{X_n}$ is equal for all $n$. Otherwise, the intensity would be a stochastic process ...


1

There are many questions here, and some of them should perhaps be split into their own posts. But to address the title question, consider the Ornstein–Uhlenbeck process, which can be defined by the SDE $dX_t = dB_t - X_t \,dt$, or in various other equivalent ways. (In Wikipedia's notation, I am using the parameters $\mu = 0$ and $\theta = \sigma = ...



Only top voted, non community-wiki answers of a minimum length are eligible