Markov chains and conditioning on events with probability $0$ Let $n \in \mathbb{N}$. Suppose we are rolling a dice, and we observe the number on the top face for each roll. Let $M_n$ be the largest number up till and including roll $n$. 
In this problem, the state space is $\mathcal{S} = \{ 1,...,6 \}$. I am wondering, how are we to interpret and work with condition on events with probability $0$. For instance, how does one define
$$ \mathbb{P}(M_3 = 2 | M_2 = 1, \ M_1 = 6) \ .$$
According to the definition of a Markov chain, this is something that should have a value, but clearly it is ill-defined. 
This problem popped up in an exercise in which we were required to show whether a number of different random variables corresponded to Markov chains. In all of the cases, I ran into the following situation where there are some chains, as above, which are ill-defined, yet
$$ \mathbb{P}(M_3 = 2 | M_2 = 1) = \frac{1}{6} \ .$$
I am sure there is something I am not understanding in the definition of the Markov chain which excludes these sorts of cases. In particular, the event where $M_2 = 1$ and $M_1 = 6$ is impossible since no such sequence of rolls exists.
 A: Consider the Markov chain defined with the following state transition matrix
$$\pi=\begin{bmatrix}
p_{i,j}
\end{bmatrix}=\begin{bmatrix}
\frac16\ \frac16\ \frac16\ \frac16\ \frac16\ \frac16\\
0\ \frac26\ \frac16\ \frac16\ \frac16\ \frac16\\
0\ 0\ \frac36\ \frac16\ \frac16\ \frac16\\
0\ 0\ 0\ \frac46\ \frac16\ \frac16\\
0\ 0\ 0\ 0\ \frac56\ \frac16\\
0\ 0\ 0\ 0\ 0\ \  1\\
\end{bmatrix}$$
where 
$$p_{i,j}=P(M_n=j\mid M_{n-1}=i)$$
and $M_n$ denotes the maximum number rolled until the $n^{th}$ trial. 
Indeed, for example, if $M_{n-1}=3$ and $M_{n}=4$ then the current state is $3$ and the next state is $4$; in order for this to occur we need to roll $4$, that is $p_{3,4}=\frac16$. However, if $ M_n=3$ then we need to roll either $1$ or $2$ or $3$. That is $p_{3,3}=\frac36=\frac12$, and obviously $p_{3,2}=p_{3,1}=0$.
Note that the $n+1^{st}$ state does not depend on the $n-1^{st}$ only on the $n^{th}$ one.
Having defined our Markov chain, we may say that we do not define $$P(M_n=i\ \mid \ M_{n-1}=j, M_{n-1}=k)$$ as
$$\frac{P(M_n=i\cap M_{n-1}=j\cap M_{n-2}=k)}{P(M_{n-1}=j\cap M_{n-2})}$$
because that would be $$\frac00.$$
A: Of course the conditional probability is ill-defined if the conditioning event has zero probability. But in what matters to us, this issue causes no problem. If $\mathbb{P}(A) = 0$, then both sides of
$$ \mathbb{P}(E\cap A) = \mathbb{P}(E\mid A)\mathbb{P}(A) $$
are identically $0$ once we have assigned any value to $\mathbb{P}(E\mid A)$. 
In general we leave them undefined as there is not much utility on it and there is no satisfying way of determining the value to be set. But when it comes to Markov chain, we do have a consistent way of assigning values and there is indeed an advantage of doing so: We need not bother whether the conditioning event has probability zero or not, hence reducing unnecessary burdens on notation and explanation. For instance, if we set $p_{i,j} = \mathbb{P}(M_n = j \mid M_{n-1} = i)$, then the Chapman-Kolmogorov equation
$$ \mathbb{P}(M_n = i_n \mid M_0 = i_0) = \sum_{i_1,\cdots,i_{n_1}} p_{i_0,i_1}p_{i_1,i_2}\cdots p_{i_{n-1},i_n} $$
becomes consistent with the law of total probability
\begin{align*}
&\mathbb{P}(M_n = i_n \mid M_0 = i_0) \\
&= \sum_{i_1,\cdots,i_{n_1}} \mathbb{P}(M_1=i_1 \mid M_0=i_0) \cdots \mathbb{P}(M_n=i_n \mid M_{n-1}=i_{n-1},\cdots,M_0=i_0)
\end{align*}
if we assign $\mathbb{P}(M_n=i_n \mid M_{n-1}=i_{n-1},\cdots,M_0=i_0) = p_{i_{n-1},i_n}$ even in the case the conditioning event has zero probability.
In summary, I would say that it is a matter of taste either to leave $\mathbb{P}(M_3=2 \mid M_2 = 1, M_1 = 6)$ undefined or to assign the value $p_{2,3}$ to this.
