# Discrepancies on using count to find Conditional and Joint Probability

Let's say my data looks as follows:

object 1: A B C

object 2: C B D

object 3: E B A

object 4: A B E

where each object is represented (not uniquely) by a combination of 3 characteristics, where characteristics $$\in$$ {A, B, C, D, E}. The ordering of characteristics does not matter.

Based on this data, I am trying to answer the following:

Given A is already present for some object, what is the most probable co occurring characteristic?

While trying to solve this, I ran into some very basic probabilistic errors as noted below:

Notations:

• $$l_{i}$$ = letter i such that $$l_{i} \in {A, B, C, D, E}$$. So for example: $$P(B|A)$$ means if A has already occurred in a row, what is the probability of B occurring in the row
• Letters cannot repeat in a row
• The three columns can take values only from {A, B, C, D, E}
• Let's assume we are working with this sample dataset of 4 objects only !!

Using: $$P(l_{i}) = \frac{count(l_{i})}{Total Count}$$

Gives:

P(A) = 3/12

P(B) = 4/12

P(C) = 2/12

P(D) = 1/12

P(E) = 2/12

Problem 1: Let's find $$P(l_{i}|A)$$ using: $$P(l_{i}|A) = \frac{rowcount(l_{i}, A)}{rowcount(A)}$$

P(A|A) = 0

P(B|A) = 1 (total 3 rows with A. B occurs in all 3)

P(C|A) = 1/3

P(D|A) = 0

P(E|A) = 2/3

This gives $$\sum P(l_{i}|A) = 2$$ (why?? should sum to 1)

Problem 2: Let's find $$P(l_{i}, A)$$ using: $$P(l_{i}, A) = \frac{count(l_{i}, A)}{ {3 \choose 2} * totalrows}$$

In 4 rows with 3 values, there are in total 12 pairs.

P(A, A) = 0

P(B, A) = 3/12 # count of cooccurrence of A and B divided by total 12 pairs

P(C, A) = 1/12

P(D, A) = 0

P(E, A) = 2/12

Now $$\sum P(l_{i}, A)$$ should be equal to P(A). However, $$\sum P(l_{i}, A) = 0.5$$, which is 2 times P(A).

What am I doing wrong??

• @Graham Kemp: Thank you for the editing. The question looks much better formatted now !! Commented Oct 12, 2017 at 15:37
• The probabilities do not sum to 1 because the events are not disjoint. For example, both B and E can co-occur with A at the same time; and do so twice, even. Commented Oct 12, 2017 at 15:40
• @Graham Kemp: Thank you for the comment. I see what you are saying with respect to events not been independent. Though I would still need to wrap my head around in order to understand it. So in this case, how do I find most probable co-occurring characteristic given A? Commented Oct 12, 2017 at 15:42
• You have it right there. Which probability is greatest out of $\mathsf P(B\mid A), \mathsf P(C\mid A), \mathsf P(D\mid A),$ or $\mathsf P(E\mid A)$? Commented Oct 12, 2017 at 15:48
• $P(B|A)$. So basically you are saying that the ordering is still correct even though I am making an incorrect assumption about independence !! Commented Oct 12, 2017 at 15:50

Don't over think this.   Just ask "which probability is greatest out of $\mathsf P(B\mid A), \mathsf P(C\mid A), \mathsf P(D\mid A),$ or $\mathsf P(E\mid A)$?"
Or even simpler, "It is obviously $B$ because $B$ occurs in every object of the sample."
• Thank you Graham !! I have three questions: 1) If I am doing ordering, can a mathematician call me out on this for doing something wrong ?? 2) I assume I can extend/generalize this to multiple events as well ex: $P(B|A,C,D)$ assuming there is no 3 characteristic limit. 3) Isn't this the same assumption of Independence that people make in Natural Language Processing language models where they assume $P(next word | given word)$ relies only on the given word? Commented Oct 12, 2017 at 15:57