Is it possible to decompose a conditional probability with three or more elements (i.e. number of events $n>=3$, where $n$ is the number of elements or events) into conditional probabilities of only two elements or the marginal probabilities of one element? Knowing this decomposition, it would help to solve higher order Markov Chain mathematically. I also know that this decomposition can be solved if we add assumption of conditional independent.

To make it concrete here is a negative example:

$Pr(c│a,b)=(Pr(a,b│c)∙Pr(c))/(Pr(a│b)∙Pr(b) )$.

Notice that the RHS still contains a conditional probability with three elements $Pr(a,b│c)$.

Assuming conditional independent on $c$, we have $Pr(a,b│c)=Pr(a│c)∙Pr(b│c)$. Thus, the conditional probability decomposition becomes

$Pr(c│a,b)≅(Pr(a│c)∙Pr(b│c)∙Pr(c))/(Pr(a│b)∙Pr(b) )$

My question is whether this type of conditional probability decomposition into one or two element is possible without making assumption. If it is really unsolvable problem, then at least we know that the assumption of conditional independent is a must.


1 Answer 1


If you make no assumption, then you might have this joint distribution (using indicators)

A   B   C   Prob
1   1   0   1/3
1   0   1   1/3
0   1   1   1/3

and all the "one element" and "two element" expressions have positive probabilities, such as $\Pr(A)=\frac23$ and $\Pr(A,B)=\frac13$ and $\Pr(A\mid B)=\frac12$, meaning their products and quotients are also positive,

but the "three element" expressions are $0$, as in $\Pr(A,B,C)=0$ and $\Pr(A,B \mid C)=0$ and $\Pr(A\mid B,C)=0$,

demonstrating that none of the "three element" expressions can be written as products and quotients of "one element" and "two element" expressions.

  • $\begingroup$ Can it be generalized to n>3 elements? $\endgroup$ Sep 15, 2022 at 8:07
  • $\begingroup$ @KardiTeknomo What kind of generalisation do you want? Usually it is not possible but for some cases (e.g. involving independence) it would be $\endgroup$
    – Henry
    Sep 15, 2022 at 9:01
  • $\begingroup$ I appreciate you just proved that for n==3 that it is basically not possible to do conditional probability decomposition without making independent assumption. My question is whether the same proof is applicable for n>3? if so, how? $\endgroup$ Sep 15, 2022 at 16:07
  • $\begingroup$ Since usually it is not possible, you could simply allocate random probabilities to the $2^n$ possible combination of events (adding up to $1$) - you would almost certainly not be able to decompose these is the way you describe $\endgroup$
    – Henry
    Sep 15, 2022 at 16:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .