0
votes
0answers
32 views

Proof of Hammersley and Clifford theorem in Besag's paper

I am reading Besag's paper on Spatial Interaction and the Statistical Analysis of Lattice Systems, see http://www.cise.ufl.edu/~anand/fa11/Besag_Spatial_interaction.pdf. In section 3, it introduces ...
0
votes
1answer
65 views

Continuous time markov chains, is this step by step example correct

I have some questions regarding CTMC... and most importantly whether the step-by-step example I provide below is correct. My main sources about CTMC are: ([1], and [2]). Let's assume 3 possible ...
0
votes
0answers
19 views

Initializing MCMC walkers with ambiguous direction (-/+)

I'm running a sampler program where there are observations given as sample data which are derived from an equal sized population of parameters that are converted to the observations using a known ...
2
votes
1answer
30 views

Markov chains by hand

If I have a starting point: $A_T=[0,1]$ at $T=1$ and a one step transition matrix of: $B=\left[ \begin{align} &\frac34 & \frac14& \\& \frac1{20}& \frac {19}{20} &\end{align} ...
1
vote
1answer
43 views

Ehrenfest urn model expectation question

Consider the Ehrenfest urn model in which $M$ molecules are distributed between two urns, and at each time point one of the molecules is chosen at random and is then removed from its urn and placed in ...
1
vote
1answer
63 views

General birth and death process

hi i need some help to understand the following (from the general birth and death process).I'll give some context first , then i ask questions. Consider general birth and death process with birth ...
3
votes
1answer
59 views

A linear growth model with immigration

Ill give some background first before asking questions.(the text below is straight out of the book) Each individual in the population is assumed give birth at an exponential rate of $\lambda$ in ...
1
vote
1answer
37 views

Probability error

I perform $N$ independent trials with $M$ successes. The probability of success is therefore $P=M/N$. Can I assign a sample-size-dependent error to the probability based only on this information? i.e. ...
0
votes
1answer
27 views

Steady state of a $4 \times 4$ transition matrix

Normally I just take $q(M_{m\times n} - I_{m\times n})$ to workout the steady state, but here I have: $$\left(\begin{array}{rrrr} 0 & 0 & .8 & .2 \\ .4 & .6 & 0 & 0 \\ .2 ...
0
votes
1answer
23 views

Limiting probability of Markov chain(Terminology)

If I am asked to find the limiting probability of a Markov chain, what does this pertain to? $\lim \limits_{n \to \infty} P^n$? Where $P$ is the stepping matrix and $n$ is the number of steps. "What ...
1
vote
1answer
91 views

Mean time spent in transient states/Markov chain

I dont get this in my book: For transient states $i$ and $j$ , let $s_{ij}$ denote the expected number of time periods that the markov chain is in state $j$ , given that it starts in state $i$. Let ...
0
votes
0answers
16 views

Markov chains - Proof of how to check recurrent states

Question 1 I read a proof of how to check recurrent states. There is one = sign that I do not understand, see the image. ...
0
votes
0answers
7 views

How can I calculate distribution of minima of sections of a continuous path (from a stochastic process)?

I have a long slab whose width is defined by a stochastic process, whose complete statistics I am aware of, say. I now cut it into smaller sections of uniform length, and calculate the minimum width ...
0
votes
0answers
20 views

Hidden Markov Model Confidence Interval (preferably in MATLAB)

I'm trying to uncover the transition parameters of data of a hidden Markov Model using MATLAB. Using the built in hmmtrain function, I can estimate the parameters quite well (I already know what they ...
2
votes
1answer
35 views

On track Prerequisite for Statistics and Probability

I do not really have a solid mathematical background because of the range of courses i had back in high school/university that wasn't really scientific oriented. Presently i am doing an MSc in ...
1
vote
1answer
47 views

Stat question about Markov Chain

An urn contains five red and three green balls. The balls are chosen at random, one by one, from the urn. If a red ball is chosen, it is removed. Any green ball that is chosen is returned to the urn. ...
4
votes
1answer
58 views

Joint Probability from Marginal Probabilities

$X, Y_1, Y_2$ are random variables with (possibly) different finite alphabets. For given conditional probability mass functions $\mathbb{P}(Y_1|X)$ and $\mathbb{P}(Y_2|X)$, is it always possible to ...
0
votes
0answers
23 views

propability of largest samples of repeated random divisions

I have a bunch of numbers $A_1=\{a_{1,1},\dots,a_{n,1}\in\mathbb R^+\}=\{1,\dots,1\}$, that get multiplied by independent uniformly $[0,1]$ distributed samples, e.g. $a_{i,2}=X_ia_{i,2}$. This process ...
0
votes
0answers
60 views

Concentration inequality for finite-state Markov chains

Many concentration inequalities characterize the fact that the arithmetic mean of a sequence of random variables converges (usually exponentially) to its expectation, such as the Chernoff bound for ...
0
votes
0answers
34 views

Discrete Time Markov Chain Probability Question

I just wanted clarification for the probability in a DTMC. I know this conditional probability with 3 variables if S = {a,b,c}: $$ P(X_1=a,X_2=b|X_3=c) = P(X_1=a|X_2=b,X_3=c)P(X_2=b|X_3=c) $$ but ...
0
votes
0answers
83 views

Markov Chains Proof using Statistics

Source: This came from "Introduction to probability" by Charles Miller Grinstead, and James Aurie Snell. It was located on page 407 and is Theorem 11.1 in the section 11.1 Introduction. Theorem: The ...
1
vote
1answer
91 views

Why Gibbs sampling needn't “remixing”

I am generating $\mathbf{x}^{(1)}, \mathbf{x}^{(2)}, \dots, \mathbf{x}^{(n)}$ using Gibbs sampling methods. So I want $\mathbf{x}^{(1)}, \mathbf{x}^{(2)}, \dots, \mathbf{x}^{(n)} \sim$ some ...
1
vote
1answer
79 views

How to Prove by definition, the given process is a Markov Process?

Define the process Xt by X0 = 1, and for t = 1, 2, . . . Xt = { uXt-1, with probability p, { vXt-1, with probability 1-p where 0 < v < 1 ...
1
vote
1answer
93 views

Markov chain transition matrix

if $P$ and $Q$ are $n \times n$ transition matrices for two Markov chain, then product $R=PQ$ is also a transition matrix. is this true ? why is it ? looks like product of transition matrix means ...
3
votes
0answers
95 views

SLLN of Markov chains .

Let $X_1$, $X_2$,... be a finite state, irreducible and aperiodic Markov chain with initial state $X_0=i$. It is known that \begin{equation} \mathrm{P}\Big\{\lim_{n\rightarrow\infty} ...
1
vote
0answers
51 views

Gibbs / MCMC sampling for sum of parameters - how to improve slow mixing?

Suppose I have a hierarchical Bayesian model, where my observational prediction, $y'$, is calculated as the sum of other parameters, ${\alpha_i}$. My observation equation (the likelihood) is: $P(y | ...
1
vote
0answers
30 views

how to describe this case with markov-chain

I want to describe this case in markov chain: The case: Mr. Meier reads NYTimes everyday and puts the newspaper on news rack. His wife sometimes cleans the house(with prob $1/3$ each day) and throws ...
0
votes
0answers
95 views

markov chain - transition matrix - why is this?

I am stuck in this trivial issue: i am given a state for which i need to give a markov chain transition matrix. i couldnot do that and i saw the solution, now i dont understand why solution this is. ...
2
votes
1answer
101 views

Confidence intervals on maximum likelihoods of observed data

I observed 400 episodes of nursing care in a hospital. I tracked the movement of the nurses between 5 rooms $A-E$. The maximum likelihood of them moving from room $i\rightarrow j$ is given by: ...
2
votes
3answers
104 views

why is this Markov Chain aperiodic

I have this Matrix: $$P=\begin{pmatrix} 0 & 1 \\ 0.3 & 0.7 \end{pmatrix}$$ this markov chain is said to be aperiodic, I dont understand how it comes to it. Period $\delta$ is the gcd of ...
0
votes
0answers
49 views

Question about Infinite Markov chains

Do 2 Markov chains $\left\{X_n\right\}^\inf_{n=0} $ and $\left\{Y_n\right\}^\inf_{n=0} $ with all of these properties exist so that the probability for infinite n values to maintain $X_n=Y_n$ is 0? ...
80
votes
9answers
22k views

How often does it happen that the oldest person alive dies?

Today, we are brought the sad news that Europe's oldest woman died. A little over a week ago the oldest person in the U.S. unfortunately died. Yesterday, the Netherlands' oldest man died peacefully. ...
0
votes
1answer
58 views

What does $P^3(D,D)$ stand for?

We're on Markov and we're considering the Markov chain on $\{A,B,C,D,E\}$ with a transition matrix $P$. I am asked to find $P^3(D,D)$ but I am unfamiliar with what the notation stands for. Thanks
0
votes
0answers
61 views

Markov Chain: Solidarity theorems

When a chain is irreducible (so each state can be reached from every other state, eventually), we quote that all states have the same character: all aperiodic / periodic with the same period, all ...
1
vote
1answer
90 views

absolute value of state space necessarily a markov chain?

Suppose $X_t$ is a first-order Markov Chain with state space $\{-1, 0, 1\}$, and transition matrix $P$, Is $|X_t|$ (absolute value) necessarily a Markov chain? Thanks!
4
votes
1answer
280 views

question involving Markov chain

Let $S_{2m}$ be the group of all permutations $\pi$ of $\{1, 2, \ldots, 2m\}$. The following transition kernel $S$ generates the random transposition walk $$ Ch(\pi, \pi')= \begin{cases} \frac{1}{2m} ...
2
votes
2answers
182 views

Markov chain basic positive recurrency question

If a discrete markov chain is stationary (as far as I know: doesn't modify itself with time), irreducible (doesn't have transient states) and aperiodic (no periodic states), is it positive recurrent? ...
1
vote
1answer
103 views

Why does this probability equivalence of events hold?

$P(X_0 = j, X_m \ne j, 1 \le m \le n-1) = P(X_m \ne j, 1 \le m \le n-1) - P(X_m \ne j, 0 \le m \le n-1) $ Where $\{X_n\}$ is an irreducible Markov Chain with a finite state space.
1
vote
1answer
62 views

On continuous Markov chains: statistics of recurrent states

Given a continuous Markov chains (and given the transition rates between the states) I would like to know the following: mean time of permanence for all states. higher order moments (i.e., ...
1
vote
0answers
39 views

Integrating with respect to an MCMC Kernel

I'm currently trying to evaluate an integral with respect to an MCMC kernel, and I wanted someone else to read what I `proved' to see if it's really correct. It just doesn't feel right. Suppose I ...
2
votes
1answer
162 views

How does the backward/forward algorithm work if there is no end?

I'm using Jason Eisner's spreadsheet to understand HMM more better. There's a box at the top that have a transition matrix. I see the Cold day and Hot day options, but don't understand why there's a ...
0
votes
1answer
76 views

A derivation problem related with Hidden Markov Chain

I encountered the problem here(Hidden Markov Chain) It is like this: The task is to compute, given the parameters of the model, the probability of a particular output sequence(observed sequence). ...
5
votes
3answers
3k views

What is the difference between all types of Markov Chains?

I have been looking for some good material covering Markov Chains but everything seems so difficult to me... After reading about the subject, I figured out that there is basically three kinds of ...