New answers tagged

1

A Markov chain can have stationary distribution $\pi(x)$ and not satisfy the detailed balance equation $\pi(x) \pi(x_p | x) = \pi(x_p) \pi(x | x_p)$. If an aperiodic and ergodic Markov chain doesn't require that $\pi(x_p | x) > 0$ for all $x_p, x$ then you can construct a counter example by having some $\pi(x_p | x) = 0$ and another $\pi(x | x_p) > 0$. ...


1

The notation is overloaded here - $\pi(x_p|x)$ is not related (directly) to $\pi(x)$, and arguably it would be clearer to replace $\pi(x_p|x)$ with some other notation, such as $q(x_p|x)$. The "constructing a Markov chain" referred to in your text means choosing a transition distribution $q(x_p|x)$ so that the stated condition holds. So you could, for ...


1

Yes: the normal distribution cares only about the distance between the mean and the point. Note that in $g(x \mid x_p)$, the mean is $x_p$ now, not $x$, so the distance between the mean in the two cases are the same and the variance in the two cases are the same. (In this symmetric situation, the algorithm is commonly called just Metropolis, rather than ...


1

The idea of the equations is conditioning on the first step. In words, the mean time to hit state 2 starting from state 1 is the sum of the mean time to hit state 2 starting from state i, times the probability to go directly to state i from state 1, plus one, to account for the first step itself. In symbols you might write something like: $$E[\tau \mid X_0=...


3

The statement $$\lim_{n\to\infty}A^n q=x$$ translates in Markov Chain World into the statement: If the Markov chain has transition matrix $A$ and the initial state $X_0$ of the chain has distribution $q$ [meaning $P(X_0=i)=q_i$], then $$ \lim_{n\to\infty} P(X_n=j) = x_j\qquad\text{for every $j$.}\tag1 $$ Since each $X_n$ lives on the same countable state ...


3

What is being said here is just convergence in distribution, and that fact is actually somewhat vacuous. The whole situation here is that you haven't specified an actual sequence of random variables, you've only specified the sequence of distributions given by $A^n q$. A Markov chain also introduces a corresponding sequence of random variables; in particular,...


1

Your problem is equivalent to toss a $2k+1$ facets die at each step, subtract $k+1$ and get the result as the $\Delta x$ to move. Equivalently you can toss a die, with $2k+1$ facets, numbered $0,\; \ldots ,\,r=2k$, and subtract $k$. Let's take this model (it simplifies the treatment). So, you are asking what is the probability that after $m$ tosses you'd get ...


0

I finally found an answer to my own question in the case where $f$ is bounded and continuous. We have that $ \frac{1}{n} \sum_{i=1}^n X^{n,i}_s$ converges in probability to $ p(\mathbb{E}_\pi f(A_0)s)$. Now if we subtract and add this term we have that $$\int_0^t f(A_{ns}) \frac{1}{n} \sum_{i=1}^n X^{n,i}_s ds =\int_0^t f(A_{ns}) (\frac{1}{n}\sum_{i=1}^n ...


0

Serval things happened. I will only consider equation 1. $$A=0.6667A+0.2222B+0.1667C$$ First I subtract $0.6667A$ from both sides so that on the LHS I get $$A-0.6667A = (1-0.6667)A = 0.3333A$$ which was rounded to $0.33A$. Moving the other two terms to the LHS and more rounding gives $$0.33A+0.22B+0.17C=0$$ which is equation 4. Similar rounding occurs in ...


0

Let $X=\{X_n:n\in\mathbb N_0\}$ be a Markov chain with state space $S$ and transition probabilities $p_{ij}$. A vector $\pi\in\mathbb R^{\#S}$ is a stationary distribution for $X$ iff $\pi_j\geqslant0$ for all $j\in S$, $\pi_j = \sum_{i\in S}\pi_j p_{ij}$ for all $j\in S$, $\sum_{j\in S} \pi_j = 1$. For each state $i\in S$, let $C_i = \{j\in S : i\...


1

Assuming that you can sample from the Dirichlet distribution efficiently, a simple approach would be to make your base rule be sampling independently from the Dirichlet distribution, then use Metropolis-Hastings acceptance/rejection. Call the Dirichlet pdf $d(\cdot;b)$. Then your acceptance probability for the transition $\alpha \to \alpha'$ is the smaller ...


3

Yes, you are right. This comes down to the fact that exponential random variables are memoryless. The idea is that a "reset" exponential has the same distribution as a conditioned one. Here's the math: Let $X$ be exponential with rate $\lambda$, and let $b > a$. Then $$\mathbb{P}(X > b | X > a) = \frac{\mathbb{P}(X > b)}{\mathbb{P}(X > a)}...


2

Don't begin by writing everything in terms of $\pi_0$. The first few terms are complicated, but for $n\geq 3$ we have the regular pattern $p\pi_{n-1}+q\pi_{n+1}=\pi_n$ which can be rewritten $p(\pi_{n-1}-\pi_n)=q(\pi_{n}-\pi_{n+1})$. By induction, $$\pi_n-\pi_{n+1}=\left({p\over q}\right)^{n-2}(\pi_2-\pi_3),$$ and therefore $$\pi_2-\pi_{n}=\sum_{j=2}^{n-1}\...


1

$$P = \begin{bmatrix}\\Row&5A0B & 4A1B & 3A2B & 2A3B & 1A4B & 0A5B \\ 5A0B & 1 & 0 & 0 &0&0 &0\\4A1B& 0.16 & 0.68 & 0.16 & 0 & 0 &0\\3A2B &0&0.24&0.52&0.24&0&0\\2A3B&0&0&0.24&0.52&0.24&0\\1A4B&0&0&0&0.16&0.68&0.16\\...


2

The slides are a bit unclear on your first point, but from the formulation "since $3$ of the $5$ individuals in the system are Species A" we can infer that birth and death are considered to occur simultaneously, and the individual selected for death has the same chance of giving birth as all others. On your second question: There are $6$ different states ...


3

That's wrong (as you can tell from the fact that it's underlined in red ;-). All states except state $4$ have period $3$. The diagram is incomplete, since there are no transitions from state $4$; the statement "State $4$ is absorbing." implies that the missing transition is from $4$ to itself with probability $1$. Yes, an absorbing state has period $1$, ...


1

Hint: $X_n$ is a Markov chain and the current state is either the same or increases by $1$, up to $k$. That means $\min(2, X_n) = X_n$ as long as $X$ is in state $1$. The only possible transition (except $1 \to 1$) is $1 \to 2$ (why?) What happens with $Y_n$ then? Can you draw it as a markov chain?


0

The transition matrix for {0,1,2...,9} is 0 1 0 0 0 .. 0 0 1 0 0 .. 0 .5 0 .5 0 .. 0 0 .5 0 .5 .. ... I ran a simple simulation in excel for {0,1,...,9}. Sample size=2500. The formula for C5 would read: =IF(C4=9;9;IF(C4=1;2;IF(C4=0;1;IF(RAND()<0.5;C4+1;C4-1)))) A run of the simulation gave me the following averages for each i, with the ...


1

For the loss of $k$ to kick in, he needs to win $k-a$ times. If he does that, he will never go broke (except maybe on the round he would quit because of the $k$ losses). He needs to win those $k-a$ within the first $2k-a$ games. So compute the chance he goes broke in less than $2k-a$ games and the expected length of a game in that scenario. This gives ...



Top 50 recent answers are included