# Machine Learning: Incomplete, Positive-Only Dataset

Reading through Lemma 1's proof on page 214 of Learning Classifiers from Only Positive and Unlabeled Data research paper.

Relevant information: consider the scenario for training a binary classifier where there is only labels for a subset of the positive examples. All the negative examples and the rest of the positive examples are unlabeled. That is, let x be an example and let $$y \in \{0, 1\}$$ be a binary label. Let $$s = \begin{cases} 1, & x \ \ \text{is labeled}, \\ 0, & \text{otherwise}. \end{cases}$$

Now, suppose that $$x$$ and $$s$$ are conditionally independent given $$y$$: $$p(s=1 \mid y=1, x) = p(s=1 \mid y=1)$$

Then,

$$p(y=1 \mid x) = \frac{1}{c} p(s=1 \mid x),$$

where $$c = p(s=1 \mid y=1)$$.

Question: They make the following deduction:

\begin{align} p(s=1 \mid x) = & p(s=1 \wedge y=1 \mid x) \\ = & p(y=1 \mid x) p(s=1 \mid y=1, x). \end{align}

First, I'm not sure how they knew to start with $$p(s=1 \mid x)$$...

Second, I understand the first deduction, i.e., $$p(s=1 \mid x) = p(s=1 \wedge y=1 \mid x)$$ (since x and s are conditionally independent and $$s=1$$ is the subset of labeled examples, which consist of only positive examples, $$y=1$$, the sample space of $$s=1$$ and $$s=1 \wedge y=1$$ is equivalent). My confusion arises when I apply the conditional probability formula:

$$p(A \cap B) = p(A \mid B) p(B).$$

Since $$A \cap B = B \cap A$$, it doesn't matter whether I substitute A for s or vice versa, i.e.,

$$p(s=1 \wedge y=1 \mid x) = p(y=1 \mid s=1, x)p(s=1 \mid x)$$

should be equivalent to

$$p(s=1 \wedge y=1 \mid x) =p(s=1 \mid y=1, x)p(y=1 \mid x)$$

but it's obviously not.