# How does the Markov measure extend to cylinders with 1 specified coordinate?

Consider a Markov Chain $(X,\mathcal{F},\mu,T)$, where $X = (1,2,\dots,n)^\mathbb{Z}$ and $T$ is the left shift and a transition matrix $P=(p_{i,j})$ and stationary distribution $\pi$ such that $\pi P = \pi$, with the Markov measure defined as:

$$\mu(\{x \in X: x_{-k} = a_{-k},\dots,x_k = a_k\}) = \pi_{a_{-k}}p_{a_{-k}a_{-k+1}}\dots p_{a_{k-1}a_{k}}$$

In all the literature I've found on the matter, it is stated that extends to the full $\sigma$-algebra by Kolmogorov's Extension theorem. My problem is, I don't understand how this extends to cylinders where only a single coordinate is specified, for eg:

$$\mu(\{x \in X: x_0=i\}) = \,?$$

Is it just $\pi_i$, $\pi_i p_{ii}$, or some summation of $p_{ij}$?

-
Which would mean $\mu(\{x \in X: x_0 = a_0\}) = \pi_{a_0} p_{a_0a_1} p_{a_{-1}a_0}$? Or maybe $\pi_{a_0} p_{a_0a_0}$? For me it becomes a bit ambiguous for $k=0$. – BallzofFury Dec 13 '12 at 15:00
But if we consider $\sum_{i=1}^n \mu(\{x \in X: x_0 = i\}) = \sum_{i=1}^n \pi_i p_{ii} = \pi_i$ while the collection $\{x \in X: x_0 = i\}$ with $i = 1,\dots,n$ forms a partition of $X$, so it should sum to 1. – BallzofFury Dec 13 '12 at 15:35
I think you're missing my point. If that would be the measure of such a partition, we would have $1 = \mu(X) = \mu(\bigcup_{i=1}^n \{x \in X: x_0=i\}) = \sum_{i=1}^n \mu(\{x \in X: x_0 = i\}) = \sum_{i=1}^n \pi_i p_{ii} = \pi_i$, which is a contradiction. – BallzofFury Dec 13 '12 at 15:47
I'm not sure what you mean, but I just want to know what $\mu(\{x \in X: x_0 = i\})$ is :( – BallzofFury Dec 13 '12 at 16:12

The definition is $$\mu\{x\in X,x_j=a_j,-k\leqslant j\leqslant k\}:=\pi_{a_{-k}}\prod_{j=-k}^{k-1}p_{a_ja_{j+1}}.$$ When $k=0$, the set of indexes of the product is empty and by convention the product is $1$, so $\mu\{x\in X,x_0=i\}=\pi_{i}$.