New answers tagged

2

The second definition allows a transition matrix to be in detailed balance with respect to a general distribution (here, $\lambda$). If the Markov chain had multiple possible steady state solutions $\{ \pi_i \}_{i\in I}$, each could be in detailed balance with $P$ (or not!). Since $P$ being in detailed balance with $\lambda$ implies that $\lambda$ is a ...


0

You are given the transition matrix for what happens on exiting permanence states.   That is not for what happens at the end of any minute; only what happens when a state change does finally occur. You are also informed that the permanence time (a count of minutes) is itself a geometric random variable with given means.   That tells you the ...


1

Let $p_n$ be the probability the $n$-th toss is a $6$. Note that $p_1=1$. We obtain a recurrence for $p_n$. The probability the $n$-th toss is not a $6$ is $1-p_n$. Given that, the probability the $n$-th toss is a $6$ is $\frac{1}{5}$. Thus $$p_{n+1}=\frac{1}{5}(1-p_n)=\frac{1}{5}-\frac{1}{5}p_n.$$ The homogeneous recurrence $p_{n+1}=-\frac{1}{5}p_n$ ...


1

The Markov Chain has 6 states labelled 1 through 6. For each state, the transition probability to each of the other states is $1/5$. So the probability transition matrix is $M=\dfrac{1}{5}(J-I)$ where $J$ is the all ones matrix, and $I$ is the identity matrix. The distribution vector for the $n$th score is given by $(1,0,0,0,0,0)M^n$, i.e. by the first row ...


2

There was likely a typo in your question, as the sum is over an index $i$ that does not appear anywhere. I think the true equation is intended to be: $$ u_b = \sum_{i=1}^{\infty} P[S_1S_2\cdots S_i \neq 0, S_i=b]$$ You can prove that by defining $N_b$ as the random number of times we hit $b$ before returning to $0$ and then taking expectations of the ...


0

The existing answer is massive overkill. Each new vector either raises the dimension by one, or it doesn't. If you've already spanned a $k$-dimensional space, $2^k$ vectors don't raise the dimension, so (in the warm-up scenario) the probability of gaining a dimension is $\left(2^n-2^k\right)/\,2^n$. Thus the expected number of steps is $$ \sum_{k=0}^{n-1}\...


3

An easy algebraic way to solve this is to first consider all of the states the player could be in, and then calculate, from bottom up, the expected time until death. The more sophisticated approach in hardmath's answer can handle any healing effects with greater ease, but is not strictly necessary here. In particular, observe that a player could only lose a ...


3

In the particular situation posited there is a possibility of the targeted character receiving $0,50,100,150$ or $200+$ damage points. Because the character initially has $195$ "hit points", once the damage exceeds that it's "game over". In discrete probability we call this an absorbing Markov chain. You have a distribution of probabilities for the ...


1

Assuming a steady state, the long term rate of increase from the arrival variables must be equal to the rate of decrease from subtracting by $Q$, so: $$ E[X] = QP[C_t \geq Q] \implies P[C_t \geq Q] = \frac{E[X]}{Q} $$ You will get such a steady state if $E[X] < Q$. More formally, define the indicator function $1\{C_t\geq Q\}= 1$ if $C_t\geq Q$, and 0 ...


-1

[Drawing out a simple case, e.g. two circles of length 2 might help visualising the problem (their product becomes a "cross").] So as OP suggests, for irreducible $Z$ we need $\forall i, i', j, j'$ $\exists n : P_{(i,j)(i',j')}^{(n)}>0$, i.e. there is some $n$ for which the $n$-step transition probability from $(i,j)$ to $(i',j')$ is nonzero (for all ...


0

If you draw the graphical representation of this Markov chain, you get $s_0 \leftrightarrow s_1 \leftrightarrow s_2 \leftrightarrow s_3 \leftrightarrow s_4$. This is a bipartite graph, since there is no possible transition within $s_0, s_2, s_4$ or $s_1, s_3$ (including e.g. $s_1 \rightarrow s_1$). Consequently, the period of the chain is 2.


3

Sketch: use irreducibility to conclude that there is only one stationary distribution. Use aperiodicity to conclude that there are no other eigenvalues of modulus $1$ other than $1$. Use Gerschgorin's theorem to conclude that there no eigenvalues of modulus larger than $1$ (maybe this is obvious). Now if the transition probability matrix were diagonalizable,...


3

Hints for (a): With $\theta$ having any distribution on $(0,1)$ and $U_{n+1}$ independently uniformly distributed on $(0,1)$, you have $\mathbb{P}[X_{n+1}=1]=\mathbb{P}[U_{n+1} \lt \theta]=\mathbb{E}[\theta]$ You can find the posterior distribution for $\theta$ in the usual way: the prior density for $\theta$ multiplied by the likelihood of the observed ...


1

There is a positive probability that after $y$ steps, we will get out of the interval (by going to the left each time, for example). Call this probability $p$. Then $T_A$ is bounded by a geometric random variable with parameter $p$. Thus it is finite a.s.


3

The one-step analysis at the end of the question is the wrong way around. You need a fixed target, and the index on $E$ should index the states from which you're trying to reach that target. But in the present case there are less cumbersome ways to get the expectation values you want. For the first question, the stationary distribution is constant by ...


1

Setting $X^i=Y^i - \frac{1}{2} d_i$ and $g_i=\frac{1}{2}d_i - t_i$ gives you $A : X^i\leq -g_i$ and $B : X^i\geq g_i$. you can rewrite : $X^i = x+\sum_{l=1}^i \xi_i$ where $\xi_i$ are i.i.d r.v. taking in your case $\frac{1}{2}$ and $-\frac{1}{2}$ values with proba $p$ and $1-p$ If I choose $\mu$ such that $pe^{\frac{\mu}{2} }+(1-p)e^{-\frac{\mu}{2}}=1$, $...


1

This corresponds to a process which can be in state $A$ or $B$; if it's in state $B$, it stays in state $B$ with probability $q$ and transitions to state $A$ with probability $p$; if it's in state $A$ it transitions to state $B$ with probability $q$ and ends with probability $p$. Consider the sum $$ \sum_{n=4}^\infty(A_n+B_n)t^{n-4}\;. $$ This is the ...


1

From your own calculations, notice that you may write $$\begin{pmatrix} A_n \\ B_n \end{pmatrix} = \begin{pmatrix} 0 & p \\ q & q \end{pmatrix} \begin{pmatrix} A_{n-1} \\ B_{n-1} \end{pmatrix} = \begin{pmatrix} 0 & p \\ q & q \end{pmatrix}^2 \begin{pmatrix} A_{n-2} \\ B_{n-2} \end{pmatrix} = \dots = \begin{pmatrix} 0 & p \\ q & q \...


0

You have misunderstood. For large enough values of $n$, you will find that $p^n(x,x)>0$ for states 12, 13 and 23. Look at this: https://www.probabilitycourse.com/chapter11/11_2_4_classification_of_states.php


0

Hint. From the relation $$ u_{n+1}=(1-a-b)u_n+b,\quad n\geq0, \tag1 $$ let's find a fixed real number $\alpha$ such that $$ \left(u_{n+1}-\alpha\right)=(1-a-b)\left(u_n-\alpha\right). \tag2 $$ Inserting $(1)$ in $(2)$ and expanding the right hand side gives $$ \color{red}{(1-a-b)u_n}+b-\alpha=\color{red}{(1-a-b)u_n}-\alpha(1-a-b) $$ that is, after ...


0

From $$P(X_{n+1}=1)=(1-a-b)P(X_n=1)+b,$$ You can conclude that \begin{align} P(X_{n+1}=1)&=(1-a-b)\left[(1-a-b)P(X_{n-1}=1)+b\right]+b\\ &=(1-a-b)^2P(X_{n-1}=1)+(1-a-b)b+b \end{align} Repeating the procedure, we have \begin{align} P(X_{n}=1)&=(1-a-b)^nP(X_0=1)+b \sum_{i=0}^{n-1}(1-a-b)^i\\ &=(1-a-b)^nP(X_0=1)+b \left[\frac{1-(1-a-b)^n}{...


0

I think the "$= \mathsf{X}$" portion is just a typo. If you drop it, everything works fine. I can see that $\bar{A}(k) \uparrow \bar{A}$, but I can't show $\bar{A} = \mathsf{X}$. To show the "if $\psi(A) > 0$, then $\varphi(\bar{A}(k)) > 0$ for some $k$", use proof by contradiction. If $\varphi(\bar{A}(k)) = 0$ for all $k$ then $\varphi(\bar{A}) = 0$ ...


0

The mistake is in the equation $p_{1,1}(n)=A+(-1)^nB$. Instead, it should be $A+(-1)^nB+0^nC$, where $0^n$ is defined to be $1$ if $n=0$ and $0$ for integers $n>0$. EDIT: By the way, this equation is derived as follows: \begin{align*} p_{1,1}(n)&=e_1^TP^ne_1 \\ &= e_1^TUD^nU^{-1}e_1\\ &= e_1^TU\begin{pmatrix}1^n & 0 & 0 & 0 \\ 0 &...


1

Let $x = [p;\ 0;\ 0;\ 1 - p]$ be a row vector. It is not hard to verify that for any $0 \leq p \leq 1$, $$ x = xP $$ where $P$ is the transition probability matrix. Note that your Markov chain is not recurrent (or persistent). There is no way out from the first state and the fourth state. Therefore, the chain does not possess a unique steady distribution.


0

Let $S=\{1,2,\ldots,m\}$. Let $$[j]=\{i\in S: i\leftrightarrow j\} $$ be the equivalence class of each state $j$ under the relation $\leftrightarrow$ and $Q_{[j]}$ the substochastic matrix composed of the rows and columns of $P$ corresponding to the states in $j$. Since the columns of $P$ sum to $1$, the columns of $Q_{[j]}$ sum to at most $1$. Therefore $Q_{...


1

State 3 is absorbing, and from state 1 the chain always moves to state 2. So to never hit state 3, the chain must either (A) start in state 1 and bounce back and forth with state 2, or (B) start in state 2 and bounce back and forth with state 1. Both of these events have probability zero: Event $A$ implies that the chain starts at $1$ and the next $2n$ ...


3

Let $$\tau=\inf\{n\geqslant 0: X_n=3\} $$ and $$\tau_i = \inf\{n\geqslant 0: X_n=3\mid X_0=i\}.$$ Conditioning on $X_1$ yields the system of equations \begin{align} \mathbb E[\tau_1] &= 1 + \mathbb E[\tau_2]\\ \mathbb E[\tau_2] &= 1+ \frac{99}{100}\mathbb E[\tau_1]\\ \mathbb E[\tau_3] &= 0, \end{align} and hence $\mathbb E[\tau_1]=200$, $\mathbb ...


0

If we know the process is in state $i$ after $k$ jumps and want to know the probability it will be in state $j$ after a further $n$ total jumps, we just need to know the probability of reaching state $j$ from state $i$ in $n$ jumps. We have $$ \begin{aligned} P({Y_{n+k}}=j\mid {Y_{k}}=i)&=P(X_{T_{n+k}}=j\mid X_{T_{k}}=i) \\ &= P\left(\text{$X$ goes ...


1

I just happened to stumble upon the answer and used http://math.stackexchange.com/help/self-answer to justify posting this answer. If we define $\bar{f}=\mathbb{E}_\pi[f_0|\mathcal{J}]$ then we have that $\bar{f}$ is $\mathcal{J}$-measurable by defintion. So the sets $A^+ := \{\omega \in \Omega: \bar{f}(\omega) > \mathbb{E}_\pi f_0\}$, $A^-:= \{\omega \...


0

The transition rates are $$ q_{ij} = \begin{cases} \lambda_1,& (i,j)\in\{(0_{\lambda_1},1_{\lambda_1}),1_{\lambda_1},2_{\lambda_1}) \}\\ \lambda_2,& (i,j)\in\{(0_{\lambda_2},1_{\lambda_2}),1_{\lambda_2},2_{\lambda_2}) \}\\ \mu,& (i,j)\in\{(1_{\lambda_1},0_{\lambda_1}),(2_{\lambda_1},1_{\lambda_1}),(1_{\lambda_2},0_{\lambda_2}),(2_{\lambda_2},1_{\...



Top 50 recent answers are included