2
$\begingroup$

I was reading about unsupervised and supervised learning in machine learning, and I came across this:

Unsupervised learning and supervised learning are not formally defined terms. The lines between them are often blurred. Many machine learning technologies can be used to perform both tasks. For example, the chain rule of probability states that for a vector $\mathbf{x} \in \mathbb{R}^n$, the joint distribution can be decomposed as

$$p(\mathbf{x}) = \prod_{i = 1}^n p(x_i \vert x_1, \dots, x_{i - 1} ).$$

This decomposition means that we can solve the ostensibly unsupervised problem of modeling $p(\mathbf{x})$ by splitting it into $n$ supervised learning problems.

Upon further research, I then came across this answer by user "kccu".

I'm confused as to how kccu's answer relates to $p(\mathbf{x}) = \prod_\limits{i = 1}^n p(x_i \vert x_1, \dots, x_{i - 1} )$. Their answer is based on $P(A \cap B) = P(A\mid B)P(B)$. If we applied this to the above case for $n = 1$, then wouldn't we have $P(\mathbf{x}) = P(x_1 \vert x_1, x_0)$? This doesn't seem to make any sense, since the vector presumably has elements $x_1, \dots, x_n$, so I'm not even sure where the $x_0$ comes from? Can someone please clarify the notation used here and the relation between what I came across and kccu's answer? Thank you.

$\endgroup$
1
$\begingroup$

Assume $\mathbf{x} = (x_1, x_2),$ i.e. $n=2.$ Then your formula gives $$ p(\mathbf{x}) = \prod_{i=1}^2 p(x_i | x_1,...,x_{i-1}) = p(x_1)\cdot p(x_2 | x_1). $$

The notation is slightly confusing. In the first term you condition on $x_1,..., x_0,$ which is to be understood as the empty set. A less confusing way to state the general formula would be $$ p(\mathbf{x}) = p(x_1)\prod_{i=2}^n p(x_i | x_1,...,x_{i-1}). $$

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.