1
vote
2answers
49 views

Convergence time of a Markov chain

We know that a regular Markov chains converges to a unique matrix. The convergence time maybe finite or infinite. My interest is in the case where the convergence time is finite. How can we accurately ...
1
vote
0answers
12 views

Birkhoff-Neumann like result for stochastic matrices?

during my research I came along a nice lemma which looks like a Birkhoff-Neumann-theorem result, but in a version for stochastic matrices. Namely, I have: Lemma. Let $M$ be a stochastic matrix, then ...
1
vote
0answers
12 views

Changes in the transition matrix of a Markov chain

In most or all Markov chain theories that I know of assumes that the transition matrix does not change over time. But what if certain changes are expected to occur at certain times in the transition ...
1
vote
0answers
28 views

References for time-inhomogeneous Markov jump processes?

In some central models in life insurance mathematics, the state of the insured is modeled using a continuous-time time-inhomogeneous Markov process with finitely many states. While many results for ...
1
vote
1answer
44 views

Markov Chain Ergodic Theorem (Proof references)

Where can I find a proof of the erogidc theorem for Markov chains that doesn't use Birkhoff? The theorem states the following : Let $(X_n)_{n\in \mathbb{N}}$ be an irreducible and positively ...
1
vote
2answers
41 views

Looking for good literature on Markov Chains with explicit calculations

I am currently starting my thesis on Markov Chains and am looking for good books and papers that include explicit calculations. I have taken a small course on Markov Chains so the subject is not ...
3
votes
1answer
69 views

When are stable continuous time Markov chains Feller? Always?

This is a question is similar to this 2 year-old one that never got answered (truthfully it's pretty much the same question except that I'm adding a bit more detail and the assumption that the $Q$ ...
0
votes
1answer
99 views

“Multivariate” Markov Chains

I am interested in estimating regime-switching VAR models to a regime setup I don't know the name of. I am hoping that someone can help me out with some references, or if there exists a name for it ...
0
votes
0answers
24 views

Reference request for Markov chains [duplicate]

I'm reading about Markov chains in Grimmett/Stirzaker, and I find the notation in that book confusing and the exposition only decent. Does anyone know a better reference?
2
votes
2answers
333 views

Expected state of a Markov chain

Let's start with a slightly trivial Markov chain defined as follows: the beginning state is called $1$ and the set of states is $\mathbb{N}$. At each step, when the current state is $n$, the ...
1
vote
0answers
31 views

literature to learn more on ergodic harris recurrent chains with an atom

I'm trying to learn more on the topic mentioned in the title. Namely I'd like to get more information on the behavior of the boundary terms. ie if I decompose sum of my chain (suppose it's ...
1
vote
1answer
86 views

Time Periodic Homogeneous Markov Chain

I want to find a textbook or survey article reference with a treatment of discrete-time, inhomogeneous, yet time periodic, markov chains on finite state spaces. Elaboration: I have an inhomogeneous ...
2
votes
0answers
158 views

Boundedness of expected reward Markov chain (may be related to discret $M/M/\infty$ queue)

[EDIT]: I read a bit on $M/M/\infty$ queue and it may not be the right comparison and my notation may be confusing (I'm in discrete time and $\lambda,\mu$ look likes rates when they are probability). ...
1
vote
0answers
51 views

Efficient random number generation for sojourn times in semi-Markov processes

I'm doing a self-study of semi-Markov processes and was wondering if there are efficient methods for generating random numbers for sojourn times. For example, generating a bunch of random numbers from ...
3
votes
0answers
32 views

Name for maximum transition probability

Let $p(x,y)$ denote the transition probability of a markov chain. Similarly, let $p^n(x,y)$ be the n-step transition probability. My question is, is there a formal name for $S(x,y):=\sup_n p^n(x,y)$. ...
1
vote
0answers
34 views

Books about Markov Models

I am looking about books on Markov chains, with recent findings such as autoregressive HMM, HMM with inputs, multiple HMM connected together. Is there anything I can look at?
1
vote
0answers
47 views

Markov chain supplementary litterature

I'm studying markov chains through Durrett and I'm finding it quite hard to read. Does anyone have a good idea to supplementary book, preferably one on his level of generality and which studies some ...
1
vote
1answer
2k views

Markov Models simple introduction

We're studying Markov models (still at the basis: transitory states, periodic states, etc..) but the professor isn't very good at teaching and I feel I'm getting lost soon. I'd love to have a simple ...
4
votes
1answer
211 views

Does every continuous time minimal Markov chain have the Feller property?

Consider a Q-matrix on a countable state space. (A Q-matrix is a matrix whose rows sum up to $0$, with nonpositive finite diagonal entries and nonnegative offdiagonal entries). As explained for ...
1
vote
0answers
51 views

Green kernel of Hunt processes

Let $X$ be a regular Hunt process on $R^+$ with starting at $x$ and $T_y :=\inf\{t>0: X_t=y\}, y\in R^+$. $G_q(\cdot,\cdot), q >0$ denotes Green kernel of the process $X$. We have the following ...
5
votes
0answers
222 views

Potential theory: discrete-time Markov processes

Recently I've found lecture notes on "Analysis on Graphs" where the potential theory methods were used to study discrete-time, time-reversible Markov chains (i.e. the state space is countable). ...
1
vote
1answer
181 views

Steady distribution for the reflected random walk

Let us consider the state space being $0,1,\dots,M$ for some $M\in \mathbb N$ and put there $N$ walkers: $$ X = (X_1,\dots,X_N). $$ Each of the walkers move independently, they can be in the same ...
1
vote
1answer
88 views

Shuffling cards and the horseshoe map

I wonder if there is a connection between the dynamics of repeated cut & shuffle operations on a deck of cards, and topological chaotic maps such as the horseshoe map? I ask this entirely naively. ...
7
votes
2answers
309 views

“Small sets” in Markov chains

I came across a definition for a "small set" (of the state space) $A \subset \Omega$: there exists a $\delta > 0$ and a measure $\mu$ such that $p^{(k)}(x, \cdot) \geq \delta \mu (\cdot)$ for every ...
4
votes
2answers
999 views

First time passage decomposition for continuous time Markov chain

For discrete time finite Markov chain, the first passage time $T_j$ to visit state $j$, is determined from the recurrence equation: $$ p^{(n)}_{ij} = \sum_{k=0}^n f_{ij}^{(k)} p^{(n-k)}_{jj} = ...
4
votes
3answers
292 views

From a deterministic discrete process to a Markov chain: conditions?

When will a probabilistic process obtained by an "abstraction" from a deterministic discrete process satisfy the Markov property? Example #1) Suppose we have some recurrence, e.g., $a_t=a^2_{t-1}$, ...
8
votes
2answers
2k views

Nice references on Markov chains/processes?

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes ...
15
votes
5answers
3k views

Good introductory book for Markov processes

Which is a good introductory book for Markov chains and Markov processes? Thank you.