0
$\begingroup$

I'm having some difficulty understanding a proof in James('Chuck') Norris book on markov chains.

Let $P$ be irreducible and aperiodic, with an invariant distribution $\pi$. Let $\left(X_n\right)_{n\geq 0}$ be a discrete time Markov$\left(\lambda,P\right)$, where $\lambda$ is the initial distribution. In this setting, Chuck wants to prove that the $X_n$ converges to an equilibrium distribution. The first part of the proof is the picture below. enter image description here

1) Theorems 1.7.7 and 1.5.7 were proven for 'univariate' markov chains($X_n$ or $Y_n$), not multivariate like $W_n$. How can we apply these theorems to $W_n$? Also, what's the meaning of an invariant distribution when $\lambda$ is of dimension $I \times I$? Do I apply the usual definition to rows and columns?

Any help would be appreciated.

$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

$W_n$ can be easily regarded as a regular (single-dimension) Markov chain. For example, say $X_n,Y_n$ share a common state space $S=\{0,1\}$. Then $W_n$ has state space $S_W=\{(0,0),\;(0,1),\;(1,0),\;(1,1)\}$. You can choose to relabel these $4$ states if you wish. So $P_W$ is a $4\times 4$ matrix and can be treated like any other Markov chain. Hence the previous theorems are valid for it.

$\endgroup$
4
  • $\begingroup$ Many thanks for your answer, Mick. Yes, I thought so, that's why I've recently asked about Cartesian Product of countable sets. Therefore, this proof is only valid for countable state-spaces. Right? Also, what about the definition of invariant distribution. How do I apply it, here? $\endgroup$
    – An old man in the sea.
    Dec 15, 2015 at 11:22
  • $\begingroup$ @Anoldmaninthesea. I haven't worked with MCs of uncountable state spaces so can't say for sure but my guess is this proof is not valid as it is, e.g. working with probability terms like $p_{ij}$ could be problematic in an uncountable state space. Defn of invariant distribution should be applied as usual: you want vector $\lambda_W$ such that $\lambda_W P_W=\lambda_W$. $\endgroup$
    – Mick A
    Dec 15, 2015 at 11:36
  • $\begingroup$ But, here, the initial distribution of $W$ is a matrix, not a row vector... $\endgroup$
    – An old man in the sea.
    Dec 15, 2015 at 21:13
  • $\begingroup$ @Anoldmaninthesea. I think is a row vector - if you think of $\mu$ as $[\mu_{00},\;\mu_{01},\;\mu_{10},\;\mu_{11}]$. That is, a vector length $4$, not a $2\times 2$ matrix. Say in the above example, the initial distributions of $X,Y$ are, respectively, $\lambda=[\lambda_0,\lambda_1],\;\pi=[\pi_0,\pi_1]$. Then that of $W$ is, via the rule $\mu_{ij}=\lambda_i\pi_j$, $$\mu=[\mu_{00},\;\mu_{01},\;\mu_{10},\;\mu_{11}]=[\lambda_0\pi_0,\;\lambda_0\pi_1,\;\lambda_1\pi_0,\;\lambda_1\pi_1]$$ $\endgroup$
    – Mick A
    Dec 15, 2015 at 21:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .