0
$\begingroup$

Let's say my data looks as follows:

object 1: A B C

object 2: C B D

object 3: E B A

object 4: A B E

where each object is represented (not uniquely) by a combination of 3 characteristics, where characteristics $\in$ {A, B, C, D, E}. The ordering of characteristics does not matter.

Based on this data, I am trying to answer the following:

Given A is already present for some object, what is the most probable co occurring characteristic?

While trying to solve this, I ran into some very basic probabilistic errors as noted below:

Notations:

  • $l_{i}$ = letter i such that $l_{i} \in {A, B, C, D, E}$. So for example: $P(B|A)$ means if A has already occurred in a row, what is the probability of B occurring in the row
  • Letters cannot repeat in a row
  • The three columns can take values only from {A, B, C, D, E}
  • Let's assume we are working with this sample dataset of 4 objects only !!

Using: $$P(l_{i}) = \frac{count(l_{i})}{Total Count}$$

Gives:

P(A) = 3/12

P(B) = 4/12

P(C) = 2/12

P(D) = 1/12

P(E) = 2/12

Problem 1: Let's find $P(l_{i}|A)$ using: $$P(l_{i}|A) = \frac{rowcount(l_{i}, A)}{rowcount(A)}$$

P(A|A) = 0

P(B|A) = 1 (total 3 rows with A. B occurs in all 3)

P(C|A) = 1/3

P(D|A) = 0

P(E|A) = 2/3

This gives $\sum P(l_{i}|A) = 2$ (why?? should sum to 1)

Problem 2: Let's find $P(l_{i}, A)$ using: $$P(l_{i}, A) = \frac{count(l_{i}, A)}{ {3 \choose 2} * totalrows} $$

In 4 rows with 3 values, there are in total 12 pairs.

P(A, A) = 0

P(B, A) = 3/12 # count of cooccurrence of A and B divided by total 12 pairs

P(C, A) = 1/12

P(D, A) = 0

P(E, A) = 2/12

Now $\sum P(l_{i}, A)$ should be equal to P(A). However, $\sum P(l_{i}, A) = 0.5$, which is 2 times P(A).

What am I doing wrong??

$\endgroup$
5
  • $\begingroup$ @Graham Kemp: Thank you for the editing. The question looks much better formatted now !! $\endgroup$ Commented Oct 12, 2017 at 15:37
  • $\begingroup$ The probabilities do not sum to 1 because the events are not disjoint. For example, both B and E can co-occur with A at the same time; and do so twice, even. $\endgroup$ Commented Oct 12, 2017 at 15:40
  • $\begingroup$ @Graham Kemp: Thank you for the comment. I see what you are saying with respect to events not been independent. Though I would still need to wrap my head around in order to understand it. So in this case, how do I find most probable co-occurring characteristic given A? $\endgroup$ Commented Oct 12, 2017 at 15:42
  • 1
    $\begingroup$ You have it right there. Which probability is greatest out of $\mathsf P(B\mid A), \mathsf P(C\mid A), \mathsf P(D\mid A), $ or $\mathsf P(E\mid A)$? $\endgroup$ Commented Oct 12, 2017 at 15:48
  • $\begingroup$ $P(B|A)$. So basically you are saying that the ordering is still correct even though I am making an incorrect assumption about independence !! $\endgroup$ Commented Oct 12, 2017 at 15:50

1 Answer 1

0
$\begingroup$

The probabilities will not sum to one because the events are not disjoint; two characteristics can (and must) co-occur with A at the same time.

Don't over think this.   Just ask "which probability is greatest out of $\mathsf P(B\mid A), \mathsf P(C\mid A), \mathsf P(D\mid A), $ or $\mathsf P(E\mid A)$?"

Or even simpler, "It is obviously $B$ because $B$ occurs in every object of the sample."

$\endgroup$
1
  • $\begingroup$ Thank you Graham !! I have three questions: 1) If I am doing ordering, can a mathematician call me out on this for doing something wrong ?? 2) I assume I can extend/generalize this to multiple events as well ex: $P(B|A,C,D)$ assuming there is no 3 characteristic limit. 3) Isn't this the same assumption of Independence that people make in Natural Language Processing language models where they assume $P(next word | given word)$ relies only on the given word? $\endgroup$ Commented Oct 12, 2017 at 15:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .