Reading through Lemma 1's proof on page 214 of Learning Classifiers from Only Positive and Unlabeled Data research paper.

Relevant information: consider the scenario for training a binary classifier where there is only labels for a subset of the positive examples. All the negative examples and the rest of the positive examples are unlabeled. That is, let x be an example and let $y \in \{0, 1\}$ be a binary label. Let $$ s = \begin{cases} 1, & x \ \ \text{is labeled}, \\ 0, & \text{otherwise}. \end{cases} $$

Now, suppose that $x$ and $s$ are conditionally independent given $y$: $$p(s=1 \mid y=1, x) = p(s=1 \mid y=1)$$


$$p(y=1 \mid x) = \frac{1}{c} p(s=1 \mid x),$$

where $c = p(s=1 \mid y=1)$.

Question: They make the following deduction:

$$ \begin{align} p(s=1 \mid x) = & p(s=1 \wedge y=1 \mid x) \\ = & p(y=1 \mid x) p(s=1 \mid y=1, x). \end{align} $$

First, I'm not sure how they knew to start with $p(s=1 \mid x)$...

Second, I understand the first deduction, i.e., $p(s=1 \mid x) = p(s=1 \wedge y=1 \mid x)$ (since x and s are conditionally independent and $s=1$ is the subset of labeled examples, which consist of only positive examples, $y=1$, the sample space of $s=1$ and $s=1 \wedge y=1$ is equivalent). My confusion arises when I apply the conditional probability formula:

$$p(A \cap B) = p(A \mid B) p(B).$$

Since $A \cap B = B \cap A$, it doesn't matter whether I substitute A for s or vice versa, i.e.,

$$p(s=1 \wedge y=1 \mid x) = p(y=1 \mid s=1, x)p(s=1 \mid x)$$

should be equivalent to

$$p(s=1 \wedge y=1 \mid x) =p(s=1 \mid y=1, x)p(y=1 \mid x)$$

but it's obviously not.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.