# Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

30 views

### Why do we have these probability functions for this Markov Chain?

The following shows one of the questions we were given in lectures a while back: We have been given the following solutions to this question: I'm rather confused by these. Take, for example, the ...
28 views

### What is an example of a second-order markov chain? [closed]

I'd like to see an example of a second-order markov chain. Haven't found one over google or in any of my textbooks
14 views

### Gaussian blur over (or random walk in) a surface mesh

Let $V$ be the set of mesh vertices, connected by edges $E$, forming a mesh that represents a surface embedded in $\mathbb{R}^3$. On this mesh a function $f:V\rightarrow\mathbb{R}$ is defined. For ...
33 views

### Transition Matrix and Invariant Probability

Given the transition matrix for a 2 state Markov Chain, how do I find the n-step transition matrix P^n? I also need to take n--> inf and find the invariant probability pi?
24 views

### Perron-Frobenius Theorem: Markov Chain -> Matrices

I am interested in finding out a way how to transform the stochastic results of perron-frobenius for markov chains to any matrix. I am aware that perron-frobenius was originally proofed with linear ...
39 views

56 views

### Finite-state Markov chain

Suppose $X_n$ is a Markov chain with transition probability matrix $p$ where the set of possible states is $S = \{1,2, \ldots, k\}$. If we are given, $X_1$, $X_2, \ldots, X_{2000}$, can we say about ...
64 views

30 views

### Building a hidden markov model with an absorbing state.

I'm working on trying to implement a hidden markov model to model the affect of a specific protein that can cut an RNA when the ribosome is translating the RNA slowly. Some brief background: The ...
144 views

19 views

### A problem on random q-colourings of a graph for randomly chosen vertex

Here is an exercise from Olle Haggstrom's "Finite Markov Chains and Algorithmic Applications" from the chapter "Fast Convergence of MCMC Algorithms". The exercise is based on random $q$-colorings of ...
22 views

### Reducing sequential correlations in Metropolis Algorithm

In our last lab, we use MCMC method to simulate a walker walking in the phase space. Using the Metropolis method, a walker at its currect position will sample another point inside a cube (centered at ...
22 views

18 views

### Markovian Model: scheduling jobs to servers

I have the following problem. I tried to look at queuing theory, but it probably fits better as a scheduling problem. I have a set of $C$ servers: each one can perform 1 job. Processes arrive ...
37 views

### Question regarding regular stochastic matrix

We say that a stochastic matrix is regular iff $\exists n\in \mathbb N$ such that $p_{ij}(n)>0$ for all states $i,j$ How many powers of a matrix do we need to compute at most in order to verify ...
41 views

### Show that a function of a markov chain is not a markov chain [closed]

How do you show that a function, $Y(n)=g(X(n))$ if some Markov chain $X(n)$ cannot be a Markov chain?
8 views

### How to find transition matrix in cascaded FSMs?

I am considering a 4-state system Equivalently, I can use a cascade approach to represent the same system as In cascade approach, If I consider independent condition on input on both subsystems,...
44 views

### For a finite state irreducible aperiodic MC, show that $P^{d^2}$ has all coordinates positive.

Suppose $X_n$ is an irreducible aperiodic finite state MC, with $P$ being the transition matrix. Then we know that $P^n$ has all positive entries for some $n\in\mathbb N$. If the state space $S$ of ...
8 views

### How to make Markov Chain model from sequence of data in MATLAB?

Markov Chain model considers only 1-step transition probabilities i.e. probability distribution of next state depends only on current state and not on previous state. I have a sequence and from that I ...
54 views

### Example of a markov chain that has a distribution that converges to some limit.

Can someone give me an example of a Markov chain that has a distribution that converges to some limit which depends on the initial distribution?
44 views

### Markov Chain that isn't Irreducible

What is an example of a Markov chain that isn't irreducible but has a unique distribution, such that its distribution converges to that unique invariant distribution for any initial distribution.
17 views

### Does an embedded discrete-time Markov chain preserve its properties in continuous time?

Given a discrete-time Markov chain without independent increments, is the embedding of it into a continuous time Markov chain (i.e. via the use of exponential waiting times) an example of a continuous ...
25 views

### How to test quality of probability estimates?

I have a Markov chain model which produces a probability distribution for absorption in 4 possible absorbing states. I.e. the model estimates the probability distribution for a discrete random ...
58 views

### Prediction Interval from Markov Chains

Thank you for taking the time to look at my question. Short, less involved question: How do you find Prediction Intervals with non-Gaussian residuals? Here is the situation: I have made a model that ...
29 views

### What does it mean for an object to not be following a definition based on some implication

I want to get a deeper understanding of what being an object that doesn't follow a definition means in terms of predicates and logical operators. Suppose the following definition of the closedness ...
73 views

### Example of a Continuous-Time Markov Process which does NOT have Independent Increments

1. Given a discrete-time Markov chain without independent increments, is the embedding of it into a continuous time Markov chain (i.e. via the use of exponential waiting times) an example of a ...
30 views

### probability of which alarm clock goes off first

I'm learning about continuous time markov chains, in the text I am reading, they are setting up the discussion by talking about a series of alarm clocks that are set: Suppose $T_1,...,T_n$ are ...
31 views

### distance from the origin in a simple random walk on $\mathbb Z^2$

let $S_{n}= \sum_{i=1}^{n}X_i$ be a simple random walk on $\mathbb{Z}$, with $S_0 = 0$. $X_i = 1$ with probability $p$ and $X_i = -1$ with probability $1-p$. It can be shown that \mathbb{P}(S_n=i\...
26 views

### Show that a matrix satisfying certain conditions is non-singular

I have a square matrix $A$ satisfying the following conditions: The elements on the diagonal are negative; All other elements are non-negative; All row sums are less than or equal to $0$; There is ...
### Markov chain - Expectation $+1$?
Let the transition matrix of a markov chain with states $\{0,1,2\}$ : A=\begin{bmatrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ 0 & \frac{1}{2} & \frac{1}{2} ...