Tag Info

Hot answers tagged

5

Of course $I-N$ is not always invertible. Clearly, if $A$ is a non-invertible function, then $N=I-A$ causes $I-N=I - (I-A) = A$ to be non-invertible. The thing is that the expression $$(I+N+N^2+\cdots)$$ makes no sense if $N$ is not some special matrix. For example, if $N=I$, what is $I+N+N^2+\dots$? It equals an infinite sum of identity matrices and it ...


2

A condition: If $N$ is nilpotent matrix, then there exists $(I - N)^{-1}$. A nilpotent matrix, for example: an upper triangle matrix with entries on diagonal are $0$.


2

The transition matrix must take into account not just the current position (that is to say, whether she is at $a$ or $b$ right now), but also the position she was at immediately before. This is because in order to know what the probability of switching positions is going to be, we must know both her current position and the position immediately preceding ...


2

Determining the stable (long term) state probabilities The solutions of the following system of equations are the stationary state probabilities: $$\begin{matrix}[\pi_A\ \pi_B \ \pi_C]\\ \\ \\ \end{matrix} \begin{bmatrix} 0.2 & 0.4 & 0.4 \\ 0.4 & 0.3 & 0.3 \\ 0.6 & 0.2 & 0.2 \\ \end{bmatrix} ...


1

Since $V(\tau) \in \{0,1\}$, we have: $$ S(t) = \int_0^t V(\tau)d\tau = \int_0^t 1\{V(\tau)=1\}d\tau $$ where $1\{V(\tau)=1\}$ is an indicator function that is 1 if $V(\tau)=1$, and 0 else. Then $\lim_{t\rightarrow\infty} S(t)/t$ is the limiting fraction of time being in state $1$, which you can work out via basic steady state theory (and the answer is ...


1

It is easier to calculate the expectation directly than it is to write down the distribution. This is usually the case. Since you haven't said that much about what you've tried yet, let me start with a hint. Try to calculate the expected time to hit $\{ 0,n \}$ starting from $x$ with $0<x<n$. Then send $n \to \infty$. You can set up a system of ...


1

Time homogeneity is not a requirement of Markov Chains. All that is needed is that the Markov Chain be memoryless. The conditional probability distribution of future values of the stochastic process given the current and past states, must be equal to the conditional probability distribution of future values of the stochastic process given the current ...


1

Regular Markov chain means that all the entries of the transition matrix $P$ are positive. Let Markov chain have $n$ states. If $\pi _i$ is the equilibrium probability for state $i$, you can say: $$ \pi _i = \sum _{j=1}^n\pi _j P_{ji} $$ Assume by contradition that $\pi _i=0$. Since all the entries of $P$ are positive, from equation above you can conclude ...


1

In order to make the entire process be Markov, you can raise the dimension, and consider a four state chain: the states are $(A,1),(A,0),(B,1),(B,0)$. (This is a common theme: you can increase the dimension to take away memory effects, or reduce the dimension at the cost of adding memory effects.) The first component represents "Alice will come in next/Bob ...


1

The transition matrix is an $N \times N$ matrix given by $\mathbf{P} = \begin{pmatrix} 1/2 & 0 & 0 & \cdots & 0 &1/2 \\ 1/2 & 1/2 & 0 &\cdots &0 &0 \\ 0 & 1/2 & 1/2 & \cdots &0&0 \\ \vdots & & \ddots & \ddots && \vdots \\ \vdots &&&\ddots&\ddots &\vdots\\ 0 ...


1

If there are currently $0$, then you either go to $N$ with probability $1/2$ or stay at $0$ with probability $1/2$. If there are currently $0<n\leq N$, then you either go to $n-1$ with probability $1/2$ or stay at $n$ with probability $1/2$. (I honestly don't know how to give just a hint for this part, since I basically just read off the content of the ...


1

The Markov chain is not ergodic. Consider a (more general) matrix of the form $$P = \begin{pmatrix} p_{11} & 0 & 0 & p_{14} \\ p_{21} & p_{22} & p_{23} & p_{24} \\ p_{31} & p_{32} & p_{33} & p_{34} \\ p_{41} & 0 & 0 & p_{44} \end{pmatrix}.$$ We are interested in $Q := P^2$. By definition (of matrix ...


1

That equality is justified by the following: \begin{eqnarray*} p(c\mid a) &=& \sum_b p(c\mid b,a)p(b\mid a) \qquad\qquad\qquad\text{(conditioning on $b$)} \\ &=& \sum_b p(c\mid b)p(b\mid a) \\ && \text{(Markov property: given $b$ we know $a,c$ are conditionally independent)} \\ && \\ \therefore\quad p(a)p(c\mid a) &=& ...


1

To your first question, the answer is generally no. The matrix $$ N= \frac 23 \pmatrix{1&1\\1&-1} $$ will have (Euclidean) operator norm $4/3 > 1$. $(I - N)^{-1}$ will exists if $\rho(N)<1$ ($\rho(N)$ denotes the spectral radius of $N$). This in turn is true if and only if $\|N\| < 1$ for some multiplicative matrix norm $\|\cdot \|$.


1

With respect to the game, this is exactly the St. Petersburg paradox. reference: http://en.wikipedia.org/wiki/St._Petersburg_paradox The game has infinite expected value but is worth only a finite amount in practice if that makes sense?


1

The answer you provided can't obey the condition that being in a state for two consecutive steps requires a different probability from when you are only in a state for one step. To describe this situation, you need to set conditional probabilities which depend on the two previous states. If the last two states were $[\dots aa]$ you transfer to $[\dots aab]$ ...


1

Firstly, the reason why exchanging rows and columns still maintains the condition we desire is because the condition does not care for where the letters are, only the number of occurrences of each letter. Swapping rows and columns clearly does not change that, so any row or column exchanges are okay. However, no matter how you switch the rows and the ...


1

Fix $n \in \mathbb{N}$ and measurable sets $B_1,\ldots,B_n$. Using the tower property, we get $$\begin{align*} \mathbb{P}(Y_1 \in B_1,\ldots,Y_n \in B_n) &= \mathbb{E} \left[ \mathbb{E} \left( \prod_{j=1}^n 1_{B_j}(Y_j) \mid \mathcal{F}_{n-1} \right) \right] \\ &= \mathbb{E} \left[ \prod_{j=1}^{n-1} 1_{B_j}(Y_j) \mathbb{E}(1_{B_n}(Y_n) \mid ...


1

Set $\DeclareMathOperator \gcd{gcd}$ $$N_x := \{n \in \mathbb{N}; p^n(x,x)>0\} \qquad \quad d_x := \gcd(N_x).$$ state 4: Since $p(4,4) = \frac{1}{2}>0$, it follows that $p^n(4,4)>0$ for all $n \in \mathbb{N}$; hence, $N_4 = \mathbb{N}$ and $d_4 = \gcd(\mathbb{N})=1$, i.e. state $4$ is aperiodic. state 3: We have $$p^2(3,3) = \mathbb{P}^3(X_2 = ...


1

Model: Set $A:=0$, $B:=50$. Let $\xi_j \sim \frac{1}{6} \delta_{-1} + \frac{1}{3} \delta_0 + \frac{1}{2} \delta_1$, $j \geq 1$, be independent identically distributed random variables. Then $$X_n := k + \sum_{j=1}^n \xi_j$$ is the position of the person after $n$ steps if the person starts at $X_0=k$. We would like to calculate the probability that the ...


1

Look at $P(N(t)-N(s)>0) \sim \mathrm{Poisson}(\lambda(t-s))$ which is related to the independent increments property. if $M \sim \mathrm{Poisson}(\lambda(t-s))$, $P(M>0) = 1-P(M=0) = 1-e^{\lambda(t-s)}$


1

I believe this approach will work for your problem: Modify the state transition matrix as follows. Since both states $0$ and $6$ are absorbing states, and you start every new run at state $1$, identify states $0$ and $6$ as state $0$, and have state $0$ go to state $1$ with probability $1$. That is, the new matrix will look like this: $$ A = \left[ ...


1

For a fixed $A\in B_\infty$, consider $\mathcal L_A:=\{B: m(A\cap B)=m(A)m(B)\}$. It suffices to show that each $\mathcal L_A$ is a $\lambda$ system, that is, $L\in\mathcal L_A$ implies that $L^c\in\mathcal L_A$ and if $L_n\in \mathcal L_A$ for each $n$, and $L_n\subset L_{n+1}$ (which is missing in the OP), then $\bigcup_{n\geqslant 1}L_n\in \mathcal ...



Only top voted, non community-wiki answers of a minimum length are eligible