-3
$\begingroup$

After reading extensively on the subject I would like to clarify this apparent problem with "Bayes Rule".

Namely the notation often used P(A and B) = P (B and A) has a big assumption that I will try to point out. This might be similar to questions regarding the inverse fallacy that I have seen around but I will try to point out something that I think is more specific although likely related.

Bayes realies on the following statement:

P (A and B) = P (B and A)

and

1) P (A and B) = P(A) * P(B|A)

"this assumes A happens first and A affects B" (i.e. the probability of A is multiplied by the probability of B once we know A has happened).

2) P (B and A) = P(B) * P(A|B)

"again, this calculates the combined probability but in terms of P(B) and the probability of A assuming B has happened (i.e. P(A|B), and therefore assumes B affects A."

This might seem logical as we use this notation because we are dealing with dependent variables. Had we been thinking these variables are independent we would say P (A and B) = P(A) * P(B).

However, my question is "What if the first assumption is correct but the second one is not?" Specifically Bayes seems to assume that if A affect B then B must affect A but this is NOT necessarily true in real life. In this case P(B and A) = P(B) * P(A) and therefore:

P(A and B) ≠ P (B and A)

I can think of several examples in which the order of events have an effect in on direction but not in the other.

So it seems to me that when using Bayes and assuming P(A and B) = P(B and A) we are making a big underlying assumption, in addition to the value estimates that will follow. Some people say well P(A and B) = L (B and A) , meaning that you are using that notation to refer to the fact that you are trying to estimate P(A and B) on the basis of a Likelihood function in terms of B, but then again that approximation could be terribly flawed (just like many things in statistics I suppose).

References: http://www.greenteapress.com/thinkbayes/html/thinkbayes002.html http://link.springer.com/article/10.3758%2FBF03195278

$\endgroup$
  • $\begingroup$ Where are your quotes taken from? $\endgroup$ – AaronS Jun 6 '16 at 7:16
  • $\begingroup$ Conditional probabilities are not about who affects who, but rather, is about the estimate of the likeliness of occurrence of $B$ given that $A$ already happens. $\endgroup$ – AaronS Jun 6 '16 at 7:20
  • 1
    $\begingroup$ "Bayes seems to assume that if A affect B then B must affect A" If "A affect(s) B" means $P(B\mid A)\ne P(B)$ then indeed "B must affect A" in the sense that $P(A\mid B)\ne P(A)$. This is a mathematical fact. If you mean something else by "A affects B", please explain what it is. $\endgroup$ – Did Jun 6 '16 at 7:22
  • $\begingroup$ @AaronS I added one of the sources. The occurrence of B given that A already happens can have a higher probability of happening than the probability of B alone, P(B), depending on what event we are talking about. However, my problem is that the opposite might not be true and some of the properties/assumptions would not be true. $\endgroup$ – BluePython Jun 6 '16 at 15:43
  • 1
    $\begingroup$ "Specifically Bayes seems to assume that if A affect B then B must affect A but this is NOT necessarily true in real life. In this case P(B and A) = P(B) * P(A) and therefore: P(A and B) ≠ P (B and A) I can think of several examples in which the order of events have an effect in on direction but not in the other." To make people understand your question, you might want to add some examples of this startling claim. $\endgroup$ – Did Jun 6 '16 at 16:03
5
$\begingroup$

You're bringing causation and temporality in where they don't belong. $A\cap B$ is merely the event which occurs if both $A$ and $B$ occur. There's no implication of temporal order or cause and effect. Events are sets, and $\cap$ has the usual set-theoretic meaning here; hence it's symmetric.

$\endgroup$
  • $\begingroup$ thanks joriki, the Theorem relies on dependent variables but it seems someone can have a variable (B) that depends on (A) but (A) might not depend or affect probability on (B). Therefore P(A|B) and the probability of P(A) would be numerically the same and that creates a problem. So in essence the dependent variables have to go both ways? IT seem to me you can have 1) independent variables 2) completely interdependent variables 3) partially independent variables. $\endgroup$ – BluePython Jun 6 '16 at 16:00
  • $\begingroup$ @BluePython: This concept of one variable "affecting" the other is misguided. There's no cause and effect here, only a symmetric relationship. You write "it seems" several times, but you don't explain why it seems to you like that. If you see an asymmetry, you're going to have to point it out more specifically. $\endgroup$ – joriki Jun 6 '16 at 16:50
  • $\begingroup$ So this symmetry is given by the fact that we often know or assume as "prior knowledge" P(A) , P(B), and we know how often we "see" A and B together, P(A and B). Therefore, our conditional probabilities in this case become "strictly" defined, since we are just solving for the unknown which can be P(A|B) or P(B|A). I guess my question ends up being "more of the same" regarding the frequentist vs Bayes debate. But it definitely seems useful to understand that P(A and B) "defines a space or a model" and therefore I can assume symmetry. $\endgroup$ – BluePython Jun 6 '16 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.