Bayesian Network/ Number of parameters

Question

Please consider the following Bayesian Network out of $Graphical Models in Applied Multivariate Statistics" by Joe Whittaker:

Now the factorization property says that the joint probability distributen $P(X_1,\ldots,X_7)$ factorizes according to the graph structure as $$ P(X_1,\ldots,X_7)=P(X_1)P(X_2|X_1)P(X_3|X_2)P(X_4|X_3)P(X_5)P(X_6|X_2,X_5)P(X_7|X_6). $$

Assuming discrete Random Variables with $r=sp(X_i)$ (here $sp(X_i)$ is the number of possible states of variable $X_i$), a total number of $$ (r-1)+(r-1)r+(r-1)r+(r-1)r+(r-1)+(r-1)r^2+(r-1)r\\=2(r-1)+4(r-1)r+(r-1)r^2 $$ parameters are required to represent $P(X_1,\ldots,X_7)$. In contrast, a non-factores representation of $P(X_1,\ldots,X_7)$ requires $r^7-1$ parameters.

---

I do not understand this last passage in grey.

Why does the factored version need $$ (r-1)+(r-1)r+(r-1)r+(r-1)r+(r-1)+(r-1)r^2+(r-1)r $$ parameters?

And why does the non-factored version need $$ r^7-1 $$ parameters?

Hope you can help me.

With greetings

Answer 1

Non-Factorised:

Say we have $3$ random variables for simplicity and that $r=2$: each variable having values $T$ or $F$. To know the full joint probability distribution, we need to know the actual probability value of each row in this table:

$$ \begin{array}{ccc|c} x_1 & x_2 & x_3 & P(X_1=x_1,\; X_2=x_2,\; X_3=x_3) \\ \hline T & T & T & 0.1 \\ T & T & F & 0.05 \\ T & F & T & 0.2 \\ T & F & F & 0.05 \\ F & T & T & 0.15 \\ F & T & F & 0.2 \\ F & F & T & 0.05\\ F & F & F & 0.2 \\ \end{array} $$

In fact, we can deduce the $8^{th}$ probability from the other $7$ because we know they all must sum to $1$.

So we need to know $2^3 - 1 = 7$ specific probability values (parameters). I hope you can see that this generalises to $r^3 - 1$ if there are $r$ choices/states for each variable, and then to $r^7-1$ if there are $7$ variables instead of $3$.

Factorised:

We need only as many probability values as required to evaluate every term on the RHS of:

$$P(X_1,\ldots,X_7)=P(X_1)P(X_2|X_1)P(X_3|X_2)P(X_4|X_3)P(X_5)P(X_6|X_2,X_5)P(X_7|X_6).$$

For $P(X_1)$ we need table:

$$ \begin{array}{l|c} x_1 & P(X_1=x_1) \\ \hline 1 & 0.1 \\ 2 & 0.05 \\ \cdots & \cdots \\ 10 \quad(r^{th} \text{ row}) & 0.02 \end{array} $$

Again, the last probability value can be deduced from the others. Therefore, for $P(X_1)$ we need $r-1$ specific probability values (parameters).

For $P(X_6 \mid X_2, X_5)$ we need table:

$$ \begin{array}{lll|c} x_2 & x_5 & x_6 & P(X_6=x_6 \mid X_2=x_2,\; X_5=x_5) \\ \hline 1 & 1 & 1 & 0.02 \\ 1 & 1 & 2 & 0.01 \\ \cdots & \cdots & \cdots & \cdots \\ 1 & 1 & 10 \quad(r^{th} \text{ row in group}) & 0.03 \\ 1 & 2 & 1 & 0.05 \\ 1 & 2 & 2 & 0.03 \\ \cdots & \cdots & \cdots & \cdots \\ 1 & 2 & 10 \quad(r^{th} \text{ row in group}) & 0.07 \\ \cdots & \cdots & \cdots & \cdots \\ \cdots & \cdots & \cdots & \cdots \\ 10 & 10 & 10 \quad(r^{th} \text{ row in group}) & 0.04 \\ \end{array} $$

Each group with the same values for $x_2$ and $x_5$ has $r$ rows. But because the probabilities for the group must sum to $1$, the last probability value in the group can be deduced from the others. So for each group we need $r-1$ probability values (parameters).

We have $r^2$ of these groups in this table because that's how many combinations of values we have for $x_2$ and $x_5$. So this table needs $(r-1)r^2$ probability values specified.

The other terms are worked out similarly and adding up all the numbers of parameters we need, we arrive at:

$$(r-1)+(r-1)r+(r-1)r+(r-1)r+(r-1)+(r-1)r^2+(r-1)r.$$