0
$\begingroup$

Reading through Lemma 1's proof on page 214 of Learning Classifiers from Only Positive and Unlabeled Data research paper.

Relevant information: consider the scenario for training a binary classifier where there is only labels for a subset of the positive examples. All the negative examples and the rest of the positive examples are unlabeled. That is, let x be an example and let $y \in \{0, 1\}$ be a binary label. Let $$ s = \begin{cases} 1, & x \ \ \text{is labeled}, \\ 0, & \text{otherwise}. \end{cases} $$

Now, suppose that $x$ and $s$ are conditionally independent given $y$: $$p(s=1 \mid y=1, x) = p(s=1 \mid y=1)$$

Then,

$$p(y=1 \mid x) = \frac{1}{c} p(s=1 \mid x),$$

where $c = p(s=1 \mid y=1)$.

Question: They make the following deduction:

$$ \begin{align} p(s=1 \mid x) = & p(s=1 \wedge y=1 \mid x) \\ = & p(y=1 \mid x) p(s=1 \mid y=1, x). \end{align} $$

First, I'm not sure how they knew to start with $p(s=1 \mid x)$...

Second, I understand the first deduction, i.e., $p(s=1 \mid x) = p(s=1 \wedge y=1 \mid x)$ (since x and s are conditionally independent and $s=1$ is the subset of labeled examples, which consist of only positive examples, $y=1$, the sample space of $s=1$ and $s=1 \wedge y=1$ is equivalent). My confusion arises when I apply the conditional probability formula:

$$p(A \cap B) = p(A \mid B) p(B).$$

Since $A \cap B = B \cap A$, it doesn't matter whether I substitute A for s or vice versa, i.e.,

$$p(s=1 \wedge y=1 \mid x) = p(y=1 \mid s=1, x)p(s=1 \mid x)$$

should be equivalent to

$$p(s=1 \wedge y=1 \mid x) =p(s=1 \mid y=1, x)p(y=1 \mid x)$$

but it's obviously not.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.