Hot answers tagged

5

You need all row sums to be $<0$, else you can do the following: $$\begin{pmatrix}-1&1&0\\1&-1&0\\0&0&-1\end{pmatrix}$$ If you assume all row sums to be $<0$, the statement is true. After multiplying by $-1$, the matrix will be a Diagonally dominant matrix.


4

Notice that we never observe $X_{n-1}$ explicitly, only $X_{n-2}X_{n-1}$ - some sort of "incomplete information" about the most recent realization $X_{n-1}$. You can construct a counterexample where $X_k\in\{-1,1\}$, i.e. knowing that $X_{n-1}=1$ gives a different probability for $Y_n=1$ than when $X_{n-1}=-1$, but in both cases, $Y_{n-1}=1$. So knowing the ...


2

The first entry in the matrix is $p_{1,1}=1/2$. It means that with $50$% probability you stay in state $1$ for one time step. To the right of $p_{1,1}$ you have $p_{1,2}$ which is the probability that you go from state $1$ to state $2$. Each entry $p_{i,j}$ means that you go from state $i$ to state $j$ where the $i$ is the row and the $j$ is the column. ...


1

$M/M/1$ and its relatives are the queues which are markov chains. This is since these queues have exponentially distributed interarrival times and service times. $M/D/1$ is not a markov chain but there exists an imbedded discrete time markov chain whose properties provide information about the process. $M/G/1, G/G/1$ etc are not markov chains. But if ...


1

There are many questions here, and some of them should perhaps be split into their own posts. But to address the title question, consider the Ornstein–Uhlenbeck process, which can be defined by the SDE $dX_t = dB_t - X_t \,dt$, or in various other equivalent ways. (In Wikipedia's notation, I am using the parameters $\mu = 0$ and $\theta = \sigma = ...


1

The process stays in state $1$ an exponential amount of time with mean $1/\lambda_1$. The process then transitions to state $1$ with probability $1/2$ or to state $2$ with probability $1/2$. Let $T$ be the time spent in state $1$ until the process reaches state $2$. As a hint, let us try to calculate $\mathbb{E}[T]$. By conditioning on the number of times ...


1

Suppose you have rolled a $4$, which gives a payout of $16$, should you roll again? The probability of getting a $j \in \{4,5,6\}$ later in the game is rolling $n$ times a $2$ or $3$ and then a $j$ for any $n \in \mathbb{N}$, which gives $$\mathbb{P}(\textrm{getting a }j)=\sum_{n=0}^\infty \frac{1}{3^n} \frac{1}{6}= \frac{1}{6} \frac{3}{2} = \frac{1}{4}.$$ ...


1

Essentials of Brownian Motion by Frank Knight has what you are looking for, and lots more.


1

This isn't an answer per se, rather some thoughts to consider. If $\{X_n:n=1,2,\ldots\}$ is a Markov chain and we define the holding times $\{T_n:n=1,2,\ldots\}$ by $T_n\sim\mathsf{Exp}(\lambda_{X_n})$, then the sequence $\{T_n\}$ only defines a Poisson process if $\lambda_{X_n}$ is equal for all $n$. Otherwise, the intensity would be a stochastic process ...


1

The stationary distribution of a continuous-time Markov chain with finitely many states is unique iff the Markov chain has only one recurrent class of states, i.e. there is only one set $S$ of states such that it is possible to go from any state $i$ in $S$ to any other state $j$ in $S$ by allowed transitions (corresponding to positive entries of the ...


1

A continuous Markov chain $X(t)$ with the state space $S$ and the generator matrix $G$ has a probability distribution $\pi$ on $S$ that is a stationary distribution for $X(t)$ if and only if it satisfies $\pi G=0$. And it is unique if the Markov chain is irreducible, i.e. if its state space is a single communicating class; in other words, if it is ...


1

Consider $\tilde B_s:=B_{s+1}$, $s\ge 0$, and notice that $R-1=\inf\{s>0:\tilde B_s=0\}=T_0(\tilde B)$. Now condition on where $B$ is at time $1$: the density of $B_1$ is $y\mapsto p_1(x,y)$ (as you surmise) and the conditional distribution of $\tilde B$, given that $B_1=y$, is $P_y$. Putting these thoughts together, $$ \eqalign{ P_x(R>1+t) ...


1

First note that $$f(x_2,t+\Delta t \ | \ x_1,t) = \frac{1}{\sqrt{2\pi \Delta t}} \exp\left\{\frac{-1}{2}\frac{(x_2-x_1)^2}{\Delta t}\right\},$$ Hence we can write (let $u = x_2-x_1$) \begin{align} &\int_{|x_2-x_1|>\epsilon} \frac{1}{\sqrt{2\pi \Delta t}} \exp\left\{\frac{-1}{2}\frac{(x_2-x_1)^2}{\Delta t}\right\}dx_2 \\ &= \int_{|u|>\epsilon} ...


1

You’re on the right track, I think. First, back up a step and label the states so that $P$ takes the form of a block cyclic permutation matrix $$P=\begin{bmatrix} 0 & P_1 & \ddots & \ddots & 0 \\ 0 & 0 & P_2 & \ddots & \ddots \\ \ddots & \ddots & \ddots & \ddots & \ddots \\ 0 & 0 & \ddots & \ddots ...


1

If $\mu^{ij}$ is nonzero, it means that there is nonzero probability that a transition occurs from state $i$ to state $j$. Each of the states is represented by a vertex in the graph (in the picture, each vertex is drawn as a box). In your example, there are four states ($G$ is a $4\times4$ matrix). Each nonzero $\mu^{ij}$ is represented as a weighted edge in ...


1

Assmussen, Applied Probability and Queues is a good place to start. Having Meyn and Tweedie's "Markov Chains and Stochastic Stability" can be useful at times as well for discrete time theory on fairly general state spaces. Norris' Markov chains and Stroock's Introduction to Markov Processes may also be handy, though they are both somewhat basic.


1

$1.$ $\textbf{Probability and Random Processes}$ by Grimmet and Stirzaker and if you also want a book with solved exercises there is a companion book that has solutions to every exercise: $2.$ $\textbf{One Thousand Exercises in Probability}$



Only top voted, non community-wiki answers of a minimum length are eligible