# Bayesian Network/ Number of parameters

Please consider the following Bayesian Network out of $Graphical Models in Applied Multivariate Statistics" by Joe Whittaker: Now the factorization property says that the joint probability distributen$P(X_1,\ldots,X_7)$factorizes according to the graph structure as $$P(X_1,\ldots,X_7)=P(X_1)P(X_2|X_1)P(X_3|X_2)P(X_4|X_3)P(X_5)P(X_6|X_2,X_5)P(X_7|X_6).$$ Assuming discrete Random Variables with$r=sp(X_i)$(here$sp(X_i)$is the number of possible states of variable$X_i$), a total number of $$(r-1)+(r-1)r+(r-1)r+(r-1)r+(r-1)+(r-1)r^2+(r-1)r\\=2(r-1)+4(r-1)r+(r-1)r^2$$ parameters are required to represent$P(X_1,\ldots,X_7)$. In contrast, a non-factores representation of$P(X_1,\ldots,X_7)$requires$r^7-1$parameters. I do not understand this last passage in grey. Why does the factored version need $$(r-1)+(r-1)r+(r-1)r+(r-1)r+(r-1)+(r-1)r^2+(r-1)r$$ parameters? And why does the non-factored version need $$r^7-1$$ parameters? Hope you can help me. With greetings ## 1 Answer Non-Factorised: Say we have$3$random variables for simplicity and that$r=2$: each variable having values$T$or$F$. To know the full joint probability distribution, we need to know the actual probability value of each row in this table: $$\begin{array}{ccc|c} x_1 & x_2 & x_3 & P(X_1=x_1,\; X_2=x_2,\; X_3=x_3) \\ \hline T & T & T & 0.1 \\ T & T & F & 0.05 \\ T & F & T & 0.2 \\ T & F & F & 0.05 \\ F & T & T & 0.15 \\ F & T & F & 0.2 \\ F & F & T & 0.05\\ F & F & F & 0.2 \\ \end{array}$$ In fact, we can deduce the$8^{th}$probability from the other$7$because we know they all must sum to$1$. So we need to know$2^3 - 1 = 7$specific probability values (parameters). I hope you can see that this generalises to$r^3 - 1$if there are$r$choices/states for each variable, and then to$r^7-1$if there are$7$variables instead of$3$. Factorised: We need only as many probability values as required to evaluate every term on the RHS of: $$P(X_1,\ldots,X_7)=P(X_1)P(X_2|X_1)P(X_3|X_2)P(X_4|X_3)P(X_5)P(X_6|X_2,X_5)P(X_7|X_6).$$ For$P(X_1)$we need table: $$\begin{array}{l|c} x_1 & P(X_1=x_1) \\ \hline 1 & 0.1 \\ 2 & 0.05 \\ \cdots & \cdots \\ 10 \quad(r^{th} \text{ row}) & 0.02 \end{array}$$ Again, the last probability value can be deduced from the others. Therefore, for$P(X_1)$we need$r-1$specific probability values (parameters). For$P(X_6 \mid X_2, X_5)$we need table: $$\begin{array}{lll|c} x_2 & x_5 & x_6 & P(X_6=x_6 \mid X_2=x_2,\; X_5=x_5) \\ \hline 1 & 1 & 1 & 0.02 \\ 1 & 1 & 2 & 0.01 \\ \cdots & \cdots & \cdots & \cdots \\ 1 & 1 & 10 \quad(r^{th} \text{ row in group}) & 0.03 \\ 1 & 2 & 1 & 0.05 \\ 1 & 2 & 2 & 0.03 \\ \cdots & \cdots & \cdots & \cdots \\ 1 & 2 & 10 \quad(r^{th} \text{ row in group}) & 0.07 \\ \cdots & \cdots & \cdots & \cdots \\ \cdots & \cdots & \cdots & \cdots \\ 10 & 10 & 10 \quad(r^{th} \text{ row in group}) & 0.04 \\ \end{array}$$ Each group with the same values for$x_2$and$x_5$has$r$rows. But because the probabilities for the group must sum to$1$, the last probability value in the group can be deduced from the others. So for each group we need$r-1$probability values (parameters). We have$r^2$of these groups in this table because that's how many combinations of values we have for$x_2$and$x_5$. So this table needs$(r-1)r^2\$ probability values specified.

The other terms are worked out similarly and adding up all the numbers of parameters we need, we arrive at:

$$(r-1)+(r-1)r+(r-1)r+(r-1)r+(r-1)+(r-1)r^2+(r-1)r.$$