# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

41 views

29 views

### The expectation of total number of different states in N time points

[Conditions] (1) An object has K possible states. (2) This object can have only one state at a single time point. (3) The probability of each state at any single time point is 1/K, and each time ...
60 views

### I cannot see what happens to $P_{ij}(s)=p_{ij}^0+\sum^\infty_{n=1}\sum^{n-1}_{k=0}s^{n-k}f_{ij}^{n-k}s^kp_{jj}^k$ to get result

This is self learning and it is stats. $P_{ij}(s)=\sum^\infty_{n=0}p_{ij}^ns^n$ (which you'll probably recognise is a generating function) and $F_{ij}(s)=\sum^\infty_{n=1}f_{ij}^ns^n$ (note n=1 ...
391 views

For an irreducible periodic (period $2$) Markov Chain I know that both of the following two quantities are same and equal to $\pi(i)$: $$\lim_{n\to \infty} \frac{1}{2}(p_n(j,i) + p_{n+1}(j,i))$$ $$\... 1answer 224 views ### If P is an invertible transition probability matrix, does P^{-1}[i,j] have any interesting meaning? Suppose we have a Markov chain transition probability matrix P that is invertible, i.e., P^{-1} exists. Question: Does there exist a meaningful interpretation of the (i,j) entry in P^{-1}? ... 1answer 947 views ### Proof about Steady-State distribution of a Markov chain Consider A as a matrix, that when normalized represents an finite-state time-homogeneous Markov chain M with entries 0\leq p_{i,j}\leq 1, where each row sums up to 1. We can also assume that ... 1answer 2k views ### Acceptance probability of Metropolis-Hastings I am an IT guy writing my masters thesis on MCMC methods for use in predicting the outcome of football(soccer) matches. Right now I am trying to wrap my head around MCMC and Metropolis-Hastings in ... 1answer 172 views ### what's the generalized approach to this infinite state markov chain problem Say, a bag has 10 balls, in which 9 are red, 1 is black. Each red ball is worth 1 point, each black is worth 4 points. I have 8 picks from the bag to start with (the bag refills itself after each ... 1answer 825 views ### Countable state Markov chain: detailed balance consequences Let S be a countable set and \pi a probability distribution on S. A discrete-time Markov chain (X_n) with state space S is said to be in detailed balance with respect to \pi (or simply in ... 1answer 1k views ### Probability distribution of markov chain I have a Markov chain with state space E = \{1,2,3,4,5\} and transition matrix below:$$ \begin{bmatrix} 1/2 & 0 & 1/2 & 0 & 0 \\ 1/3 & 2/3 & 0 &...
It is known that for a regular Markov matrix $M,$ $M^{n}$ has the steady-state vector as all of its columns as $n \to \infty.$ I learned this in class, but what if there is more than one steady-state ...