Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

60 views

Given irreducible transition matrix $P$, why does the matrix $(P−I|\mathbb{1})(P−I|\mathbb{1})^T$ have full rank?

Given irreducible transition matrix $P$, why does the matrix $(P−I|\mathbb{1})(P−I|\mathbb{1})^T$ have full rank? I know this is partially due to the fact that since $P$ is irreducible, there exists ...
33 views

30 views

Series with Markov Chains Probabilities

Notation For each $t \in \mathbb{N}$, let $h_t \in H$ be a random variable that follows a Markov chain, and $h^t \equiv \{h_0,h_1,\dots,h_t\} \in H^t$. Let $\Pi(h^{t})$ be the probability that a ...
8 views

Ergodicity coefficient of block matrix

I have a stochastic matrix of the following form $$X=\begin{bmatrix}A/3&B/3&C/3\\I_n&&\\&I_n&&\\\end{bmatrix},$$ where $A,B,C$ are all $n$ by $n$...
29 views

66 views

Markov Chain Transition Matrix Question

Ok, so my question is pretty simple, the question states: A spider web is only big enough to hold 2 flies at a time. Assuming that the flies fly into the web independently: -The probability that no ...
22 views

Strong Markov Property for Markov Chains - Statement Verification

I suspect that my handwritten lecture notes for the Strong Markov Property are wrong. I'd appreciate corrections to them. We first define the following: A random variable $\tau$ is called a ...
9 views

Markov Chain: Aperiodicity => Primitivity

Hellooo, I would like to know how I can show that the transition Matrix $P$ of an aperiodic Markov chain is primitive. Any suggestions?
15 views

Markov chain nulls

hope the question is ok for this forum. I am a developer and not a mathematician but realise your group is likely to know the answer for these questions. The background is that I am writing a program ...
30 views

Limiting Distribution of a Gibbs Distribution

I know that the Gibbs distribution at a particular state, x, is given by $\frac{e^{-\beta E_i}}{\sum_j e^{-\beta E_j}}$ with $\beta = \frac{1}{T}$, but I do not understand what a limiting distribution ...
67 views

A Question Regarding Markov Chains and Ergodicity

Suppose the Markov chain with Probability Transition Matrix, $P$ = ($p{_x}{_y}$) is ergodic and $p{_m}(x, y) > 0$ for all states $x$ and $y$. If $n ≥ m$, show that $p_n(x, y) > 0$ for all ...
32 views

Finding Initial state vector with given values

I am not sure how to use a given values to form a initial state vector. There are 30% of customers from Company A switch to company B every month, and 35% customers from Company B switch to company A ...
24 views

Correlation Matrix Question

Why is this not a possible correlation matrix for any three random variables X, Y, and Z? $\begin{pmatrix} 1 & -1 & -1 \\ -1 & 1 & -1\\ -1 & -1 & 1\end{pmatrix}$
893 views

78 views

Distribution of a process dependent on a Markov chain's states

Consider a Markov chain $X_t$ with state space $\{0,1\}$, initial distribution $$\begin{array}{l} \mathbf{P}(X_0=1)=\lambda \\ \mathbf{P}(X_0=0)=1-\lambda \end{array}$$ and transition ...
40 views

Probability that two random walks on $\mathbb{Z}^2$ meet at the origin

Suppose $X,Y$ are symmetric, independent random walks on the lattice $\mathbb{Z}^2$. I am trying to find the probability: $$\mathbb{P}\big(X_n=Y_n=(0,0)\;\text{for some}\;n\,\big|\,X_0=Y_0=(0,N)\big)$$...
23 views

MCMC and Metropolis-Hastings problem(s)

What does it mean for a particular state to be a "ground" state or a "stable" state? I should make clear that this is final exam review material and not homework. Also, how does one compute a Gibbs ...
33 views

Transition Matrix and Invariant Probability

Given the transition matrix for a 2 state Markov Chain, how do I find the n-step transition matrix P^n? I also need to take n--> inf and find the invariant probability pi?
20 views

24 views

Perron-Frobenius Theorem: Markov Chain -> Matrices

I am interested in finding out a way how to transform the stochastic results of perron-frobenius for markov chains to any matrix. I am aware that perron-frobenius was originally proofed with linear ...
19 views

59 views

8 views

Recurrent states - proof of claim

I want to prove: If $x↔y$, then $x$ is recurrent iff $y$ is recurrent. $i\in S$ is recurrent if $P(T_i<\infty)=1$ How can I properly prove this? I don't know where to start from. Thanks
23 views

Recurrence of a state in a finite state space

Suppose $T_A := \inf\{ n \ge 1 : X_n \in A\}$ where $A \subset \mathcal{S}$ is finite. Assume $\mathbb{P}\{T_A < \infty \; | \; X_0 = x\}= 1$ for $\forall x \in \mathcal{S}-A$. I need to show that ...
29 views

Recurrence of 0 in a random walk

Assume $\mathcal{S} := \{0, 1, \cdots \}$, $p(0,1)=1$ and $p(n,0)=p(n,n+1)=\frac{1}{2}$ for $n=1,2, \cdots$. Is $0$ recurrent or transient? So, basically this is an irreducible, closed but infinite ...
15 views

435 views

Figuring out probabilities with Hidden Markov Models

I'm really new to Math so sorry in advance if this question does not make sense. Also I cross posted this on stats.stackexchange.com also. Background: I'm trying to learn about hidden Markov models ...