0
votes
1answer
71 views

How to make sense out of this: Ergodic theorem for Markov chains

We had the ergodic theorem for Markov chains, stating that: For a state space $S \subset \mathbb{N}$ and all functions $f \in L^1$ (meaning that $\sum_{s \in S} |f(s)|\pi(s) < \infty$) and an ...
0
votes
1answer
19 views

Return Lemma MC

If a Markov chain is $\phi$-irreducible and has stationary distribution $\pi$, then $\phi\ll \pi$, Proof: We use the irreducibility of the chain to write the state space $E = \bigcup_{n,m \in ...
0
votes
1answer
28 views

Integration with respect to conditional measure?

Let $(X_n)$ be a Markov chain. For $i\in S$ my text defines $$N_i:=\sum_{n=0}^\infty \mathbf 1_{\{ X_n=i \}}$$ and then, as a part of a larger proof, claims that $$\mathbb E_i(N_i)=\sum_{n=0}^\infty ...
1
vote
0answers
33 views

Finding a probability measure

Could someone helpme with this problem? First, consider the transition kernel in $\mathbb{R^2}\times B(\mathbb{R})$ given by $K(x,A)=U_{S^1}(A-x)$. We can than define an other kernel in ...
0
votes
0answers
37 views

Two Markov chains: Optimal choice of initial distributions

Consider two Markov chains on the state space $X = [1;n] = \{1,2,\dots,n\}$ given by stochastic matrices $P$ and $Q$. Let $\alpha$ be the initial distribution for the first Markov Chain and $\beta$ is ...
2
votes
2answers
145 views

Proof of (Strong) Markov Property using sigma-algebras

I would like to ask if any of you know of a good resource containing rigorous proof (using sigma-algebras) of Markov Property and Strong Markov Property respectively in terms of Discrete Time Markov ...
2
votes
0answers
19 views

Problem with the uniform transience

Let $X$ be a Borel space and let us consider a Markov Chain $(\Phi_n)_{n\geq 0}$ on this space given by the stochastic kernel $$ P(x,\mathrm dy) = p(x,y)\mu(\mathrm dy) $$ where the density $p$ is ...
0
votes
1answer
41 views

Characterizing the Dependence Structure of a Rewards for a Finite State Homogenous Markov Chain

Let $\{X_n, n\geq 1\}$ be a finite state homogenous Markov chain with states $i = 1, \ldots, N$ . Let $g$ denote a function which returns out a reward for any given state of the Markov chain. Let ...