Given the graphical model of a probability distribution, either a Markov Random field (MRF) or a Bayesian Net (BN), how does one count the maximal number of degrees of freedom for all probability functions compatible with (i.e. with at least the set of indepencies encoded in) the graphical model?
For example, consider the MRF of three random variables $X_1 \leftrightarrow X_2 \leftrightarrow X_3$, where each $X_i \in S_i$ can take one of $D_i = |S_i|$ possible values. Here are two constructive attempts at counting the number of degrees of freedom $\#\mathrm{DOF}$, which disagree with eachother. Each time, we count the maximal number of degrees of freedom in each conditional probability and sum them up:
- $P(X_1,X_2,X_3) = P(X_1) P(X_2|X_1) P(X_3|X_2)$ gives $\# \mathrm{DOF} = D_1 + D_1 D_2 + D_3 D_2$
- $P(X_1,X_2,X_3) = P(X_2) P(X_1|X_2) P(X_3|X_2)$ gives $\# \mathrm{DOF} = D_2 + D_1 D_2 + D_3 D_2$
Why do the two disagree? Is $\#\mathrm{DOF}$ ill-defined, or do the two decompositions not correctly encode the exact same set of independencies from the graphical model?