Tag Info

New answers tagged markov-chains

0

Let $p$ be the probability that he eats pizza "this" week.   Then the probabilities of eating pizza or burritos next week is determined by the matrix: \begin{align}\mathrm{P S} & =\dfrac 1 6 \begin{bmatrix} 3 & 2 \\ 3 & 4 \end{bmatrix}\begin{bmatrix}p \\ (1-p)\end{bmatrix} \\ & = \dfrac 1 6\begin{bmatrix}2+p \\ ... 1 I'm going to assume that "being tied" is considered losing, thus having that the team loses 40% of its games if the second team scores the first goal. Then we have H_1={the team scored the first goal} and H_2={the opponent scored the first goal}. A={the team won}. P(A)=\sum_{i\in I}P(A|H_i)P(H_i)=0.6*0.3+0.4*0.7 $$1 Assuming one of the two teams always scores a goal, the probability of winning is the probability of each of the two mutually exclusive and exhaustive possibilities (i.e., scoring first or scoring second) times the probability of winning each way. In other words,$$\begin{align}\mathsf P(\textrm{winning}) & = \mathsf P(\textrm{first}) \cdot \mathsf ...

0

Lets define the transition matrix of the embedded chain $\Pi$, and a subset of this matrix, $\Pi_{3}$ in the following way: $$\Pi=\left(\begin{array}{cc|ccc} 0 & p_{12} & &p_{1\bullet}&\\ p_{21} & 0 & & p_{2\bullet}& \\\hline & & & &\\ p_{\bullet1} & p_{\bullet2} & & \Pi_3&\\ & & ... 0 I have to admit that I don't have the answer, it is just an educated guess: I just recommended to look into the proofs for uniformization, because there you also solve for the fixed-point of a dynamical system (CTMC) via linearization (DTMC). I think it might be possible to use uniformization also for non-homogeneous Markov chains, as in your case, but there ... 0 \max\{0.7, 0.65\} = 0.7. The maximum takes the greatest value of the entries. So your final answer is 0.3 + 0.7 = 1, of course. 0 It depends on W. For example if W=f(Y), then you are right, however if W is independent of Y, then conditioning on W can increase the conditional mutual information I(X;Z|YW)! For example if W is mod 2 sum of X,Z which is independent of X,Z then I(X;Z|Y)=0, I(X;Z|YW)=1 which violates the desired Markov chain. 1 There can be two interpretations of your question. Under the first, you are interested in the maximal population existing at any given time, i.e., \max_i Y_i, where Y_i is the population at time i. Under the second, you are interested in what is knows as total progeny, which is \sum_i Y_i. Lindvall showed that for a critical branching process, ... 0 Thankfully, YES :) You can very simply define the needed probability transition by : $$P : x, \mathcal{A} \mapsto P_0[X_1 \in \mathcal{A} | X_0=x]$$ You can then deduce from the definition of the family (P_n) that : $$P_n ( x, \mathcal{A} ) = P_0[X_{n+1} \in \mathcal{A} | X_0=x]$$ ! Then verify ... 1 Yes, your proof is correct. Alternative argumentation: For$$P^n := \underbrace{P \cdots P}_{\text{$n$ times}} = (p_{ij}^{(n)})_{i,j \in E}, \qquad n \in \mathbb{N},$$we have P^{k+l} = P^k \cdot P^l. Therefore,$$p^{(k+l)}_{ii} = \sum_{j \in E} p_{ij}^{(k)} p_{ji}^{(l)} \geq p_{i j_0}^{(k)} p_{j_0 i}^{(l)}$$for any fixed j_0 \in E. 1 I do not know which state does each row and column of the Markov Chain represents. However, you already have the transition matrix, which is irreducible and aperiodic, so you just need to solve the equation \pi P = \pi where \pi is a row vector, corresponding to each row of the matrix, then \lim_{n\rightarrow\infty}P(X_n=0)=\pi_{TTT} You should ... 1 EDIT: found an error, fixed. Hm, seems that I lied to you. We have 2 principal eigenvalues here, 1 and -1. Here's the output of python (check that I'm not mistaken in the matrix): >>> import numpy >>> numpy.linalg.eig(numpy.array([[0, 1.0/6, 1.0/3, 1.0/2, 0, 0, 0, 0], [1.0/6, 0, 0, 0, 1.0/3, 1.0/2, 0, 0], [1.0/3, 0, 0, 0, 1.0/6, 0, ... 2 Let X_n be the number of white balls in poll 1. Then X_n\in\{0,1,2,3\}. The probability transition matrix is$$ \begin{bmatrix} 0 & 1 & 0 & 0 \\ 1/9 & 4/9 & 4/9 & 0 \\ 0 & 4/9 & 4/9 & 1/9 \\ 0 & 0 & 1 & 0 \\ \end{bmatrix} $$How is this derived? Take the ... 1 Based on your attempt, you seem to want to know the probability that you arrive to state 3 from either state 2 or state 1. Since this is an absorbing chain, you will need to break out the transient portion M' from the absorbing portion:$$M':=\left(\begin{array}{c} 0.5/.8 & 0.3/.8\\ 0.1/.5 & 0.4/.5 \end{array}\right) = ...

0

I found some relevant results in Kemeny & Snell's book "Finite Markov Chains", Chapter IV, Theorem 4.3.4. Define $$Z:=I+\sum_{i=1}^\infty (P^i-A)$$ where $I$ is the identity matrix, $P$ is the transition probability matrix, and $A$ is a matrix with the stationary distribution $s$ in each row. Then, assume that you start the chain with an initial ...

1

Actually, your friend is right, the chain is not ergodic. By definition, a Markov chain with transition matrix $P$ is ergodic if there exists $n \geq 1$ such that $$P^n > 0. \tag{1}$$ A direct calculation shows that $P^n$ either equals $P$ or $$\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$$ for any $n \geq 1$. This means that $(1)$ doesn't ...

0

We have - being more explicit with the $p(x_i, y_j)$ terms, which are shorthand for $\def\Pr{\mathbb P}\Pr(X=x_i, Y=y_j)$, that $$\def\p#1#2{\Pr(X_2 = #1, X_1 = #2) = \Pr(X_2 = #1 \mid X_1 = #2) \Pr(X_1 = #2)}\p ab$$ The first term can be read off from the matrix, the second one from $X_1$'s distribution. Let's start - we call the two states $0$ and ...

0

So this is a MC with $2$ states $a$ and $b$. You should start by understanding $$p(X_2 = b | X_1 = a)$$ etc.

1

First work out the state space. It's built into the setup that the state space contains $\{ 0,1,2,3 \}$, and in fact it doesn't contain anything else, as you can check by checking that if $X_n \in \{ 0,1,2,3 \}$ then no matter what $Y_n$ is, you will have $X_{n+1} \in \{ 0,1,2,3 \}$. From there just build the transition matrix case by case. (Here I use the ...

1

The state space is $\{0,1,2,3\}$. When at state $0$, the next state has the distribution $\mathcal{Y}$ which is the distribution of $Y_n$. When at state $1$, the next state also has the distribution $\mathcal{Y}$, while at state $2$, it has the distribution $\mathcal{Y}+1$. Finally, at state $3$, it has distribution $3-\mathcal{Y}$. All in all, we obtain the ...

-1

The question can not be answered in general, i.e. there are no "formulas" to compute it. The reason being is that \begin{align*} A(D+E,i) & = P(i | D \cup E ) \\ & = \frac{P(i | D) P(D) + P(i | E) P(E)}{P(D)+P(E)}\\ & = \frac{P(D)}{P(D)+P(E)} A(D,i) + \frac{P(E)}{P(D)+P(E)} A(E,i) \end{align*} and the right hand side depends essentially on ...

0

For a fixed $i$, let $\psi(j) = \mathbb{E}_j T_i$, i.e. the expected time to reach state $i$ starting from state $j$. Then for $\psi(j)$ the following holds: $$\psi(j) = \begin{cases} 0\text{,} & j = i\\ 1 + \sum\limits_{k \neq j} p_{j k}\psi(k)\text{,} & j \neq i \end{cases}$$ where $p_{j k}$ denotes the one-step transition probability from $j$ ...

1

The equality $$\mathbb{E}\left[ {\frac{d} {{dt}}\mathbb{P}\left( {A|{X_t}} \right)|B} \right] = \sum\limits_{x \in S} {\mathbb{P}\left( {{X_t} = x|B} \right)\frac{d} {{dt}}\mathbb{P}\left( {A|{X_t} = x} \right)}$$ is wrong since $$\mathbb{P}\left( {A|{X_t}} \right) = \sum\limits_{x \in S} {\mathbb{P}\left( {A|{X_t} = x} \right){1_{\left\{ {{X_t} = x} ... 0 I think I found an appropriate result in Feller's probability book: The probability that at epoch 2n you return to the origin is \mathbb{P}(S_{2n}=0)={2n \choose n}2^{-2n} (and now I see how you come to your conjecture). Then, Lemma 1 on page 76 states that the probability that no return to the origin occurs up to and including epoch 2n is the same ... 2 I think you can simplify things a lot by just using two states, foot (state 1) and bicycle (state 2). Then you have the transition matrix:$$\begin{pmatrix} 0.2 & 0.8 \\ 0.3 & 0.7 \end{pmatrix}$$In fact, the way you've done it is a bit confusing because falling off a bicycle is not a state in the same sense that going to school by foot/bicycle ... 0 If we take \mu to be the distribution of X_0, then \sigma(X_0) depends on \mu already: \sigma(X_0) is the smallest sigma-algebra for which X_0 is measurable. Assume X_0\equiv 1 (\mu is a point measure at 1), without loss of generality. Then \sigma(X_0)=\{\Omega,\emptyset\}. Assume X_0\in\{1,2\}, and A=X_0^{-1}(1), then \sigma(X_0) ... 1 Let's look at your weather example: Let X_n be the wether on the day n. The weather it's not exactly a Markov chain, but we'll suppose that it is. Let 1=Rainy and 2=Sunny. How would you interpret$$\begin{array}{ccc}&\boldsymbol 1&\boldsymbol 2\\ \boldsymbol 1&.6&.4\\\boldsymbol 2&.2&.8\end{array} \ \ \ \ ? Does ...

1

WARNING: This is post may seem long, but I just presented the answer in a way that I thought would be most easy to read. Hope it helps clear up some confusion. I'm not sure how you're classifying your states so for clarity by Xn = RS I mean Wn-1 = R and Wn = S. Now to map the behavior of the weather based on given information we can say: P(Wn = R) = 0.3 ...

Top 50 recent answers are included