Hot answers tagged

3

The one-step analysis at the end of the question is the wrong way around. You need a fixed target, and the index on $E$ should index the states from which you're trying to reach that target. But in the present case there are less cumbersome ways to get the expectation values you want. For the first question, the stationary distribution is constant by ...


3

Sketch: use irreducibility to conclude that there is only one stationary distribution. Use aperiodicity to conclude that there are no other eigenvalues of modulus $1$ other than $1$. Use Gerschgorin's theorem to conclude that there no eigenvalues of modulus larger than $1$ (maybe this is obvious). Now if the transition probability matrix were diagonalizable,...


3

Hints for (a): With $\theta$ having any distribution on $(0,1)$ and $U_{n+1}$ independently uniformly distributed on $(0,1)$, you have $\mathbb{P}[X_{n+1}=1]=\mathbb{P}[U_{n+1} \lt \theta]=\mathbb{E}[\theta]$ You can find the posterior distribution for $\theta$ in the usual way: the prior density for $\theta$ multiplied by the likelihood of the observed ...


3

An easy algebraic way to solve this is to first consider all of the states the player could be in, and then calculate, from bottom up, the expected time until death. The more sophisticated approach in hardmath's answer can handle any healing effects with greater ease, but is not strictly necessary here. In particular, observe that a player could only lose a ...


3

In the particular situation posited there is a possibility of the targeted character receiving $0,50,100,150$ or $200+$ damage points. Because the character initially has $195$ "hit points", once the damage exceeds that it's "game over". In discrete probability we call this an absorbing Markov chain. You have a distribution of probabilities for the ...


3

Let $$\tau=\inf\{n\geqslant 0: X_n=3\} $$ and $$\tau_i = \inf\{n\geqslant 0: X_n=3\mid X_0=i\}.$$ Conditioning on $X_1$ yields the system of equations \begin{align} \mathbb E[\tau_1] &= 1 + \mathbb E[\tau_2]\\ \mathbb E[\tau_2] &= 1+ \frac{99}{100}\mathbb E[\tau_1]\\ \mathbb E[\tau_3] &= 0, \end{align} and hence $\mathbb E[\tau_1]=200$, $\mathbb ...


2

There was likely a typo in your question, as the sum is over an index $i$ that does not appear anywhere. I think the true equation is intended to be: $$ u_b = \sum_{i=1}^{\infty} P[S_1S_2\cdots S_i \neq 0, S_i=b]$$ You can prove that by defining $N_b$ as the random number of times we hit $b$ before returning to $0$ and then taking expectations of the ...


2

The second definition allows a transition matrix to be in detailed balance with respect to a general distribution (here, $\lambda$). If the Markov chain had multiple possible steady state solutions $\{ \pi_i \}_{i\in I}$, each could be in detailed balance with $P$ (or not!). Since $P$ being in detailed balance with $\lambda$ implies that $\lambda$ is a ...


1

For the loss of $k$ to kick in, he needs to win $k-a$ times. If he does that, he will never go broke (except maybe on the round he would quit because of the $k$ losses). He needs to win those $k-a$ within the first $2k-a$ games. So compute the chance he goes broke in less than $2k-a$ games and the expected length of a game in that scenario. This gives ...


1

Let $p_n$ be the probability the $n$-th toss is a $6$. Note that $p_1=1$. We obtain a recurrence for $p_n$. The probability the $n$-th toss is not a $6$ is $1-p_n$. Given that, the probability the $n$-th toss is a $6$ is $\frac{1}{5}$. Thus $$p_{n+1}=\frac{1}{5}(1-p_n)=\frac{1}{5}-\frac{1}{5}p_n.$$ The homogeneous recurrence $p_{n+1}=-\frac{1}{5}p_n$ ...


1

The Markov Chain has 6 states labelled 1 through 6. For each state, the transition probability to each of the other states is $1/5$. So the probability transition matrix is $M=\dfrac{1}{5}(J-I)$ where $J$ is the all ones matrix, and $I$ is the identity matrix. The distribution vector for the $n$th score is given by $(1,0,0,0,0,0)M^n$, i.e. by the first row ...


1

Assuming a steady state, the long term rate of increase from the arrival variables must be equal to the rate of decrease from subtracting by $Q$, so: $$ E[X] = QP[C_t \geq Q] \implies P[C_t \geq Q] = \frac{E[X]}{Q} $$ You will get such a steady state if $E[X] < Q$. More formally, define the indicator function $1\{C_t\geq Q\}= 1$ if $C_t\geq Q$, and 0 ...


1

There is a positive probability that after $y$ steps, we will get out of the interval (by going to the left each time, for example). Call this probability $p$. Then $T_A$ is bounded by a geometric random variable with parameter $p$. Thus it is finite a.s.


1

Setting $X^i=Y^i - \frac{1}{2} d_i$ and $g_i=\frac{1}{2}d_i - t_i$ gives you $A : X^i\leq -g_i$ and $B : X^i\geq g_i$. you can rewrite : $X^i = x+\sum_{l=1}^i \xi_i$ where $\xi_i$ are i.i.d r.v. taking in your case $\frac{1}{2}$ and $-\frac{1}{2}$ values with proba $p$ and $1-p$ If I choose $\mu$ such that $pe^{\frac{\mu}{2} }+(1-p)e^{-\frac{\mu}{2}}=1$, $...


1

This corresponds to a process which can be in state $A$ or $B$; if it's in state $B$, it stays in state $B$ with probability $q$ and transitions to state $A$ with probability $p$; if it's in state $A$ it transitions to state $B$ with probability $q$ and ends with probability $p$. Consider the sum $$ \sum_{n=4}^\infty(A_n+B_n)t^{n-4}\;. $$ This is the ...


1

From your own calculations, notice that you may write $$\begin{pmatrix} A_n \\ B_n \end{pmatrix} = \begin{pmatrix} 0 & p \\ q & q \end{pmatrix} \begin{pmatrix} A_{n-1} \\ B_{n-1} \end{pmatrix} = \begin{pmatrix} 0 & p \\ q & q \end{pmatrix}^2 \begin{pmatrix} A_{n-2} \\ B_{n-2} \end{pmatrix} = \dots = \begin{pmatrix} 0 & p \\ q & q \...


1

Let $x = [p;\ 0;\ 0;\ 1 - p]$ be a row vector. It is not hard to verify that for any $0 \leq p \leq 1$, $$ x = xP $$ where $P$ is the transition probability matrix. Note that your Markov chain is not recurrent (or persistent). There is no way out from the first state and the fourth state. Therefore, the chain does not possess a unique steady distribution.


1

State 3 is absorbing, and from state 1 the chain always moves to state 2. So to never hit state 3, the chain must either (A) start in state 1 and bounce back and forth with state 2, or (B) start in state 2 and bounce back and forth with state 1. Both of these events have probability zero: Event $A$ implies that the chain starts at $1$ and the next $2n$ ...



Only top voted, non community-wiki answers of a minimum length are eligible