28
$\begingroup$

I'm reviewing some notes regarding probability, and the section regarding Conditional Probability gives the following example:

$P(X,Y|Z)=\frac{P(Z|X,Y)P(X,Y)}{P(Z)}=\frac{P(Y,Z|X)P(X)}{P(Z)}$

The middle expression is clearly just the application of Bayes' Theorem, but I can't see how the third expression is equal to the second. Can someone please clarify how the two are equal?

$\endgroup$
30
$\begingroup$

We know $$P(X,Y)=P(X)P(Y|X)$$ and $$P(Y,Z|X)=P(Y|X)P(Z|X,Y)$$ (to understand this, note that if you ignore the fact that everything is conditioned on $X$ then it is just like the first example).

Therefore \begin{align*} P(Z|X,Y)P(X,Y)&=P(Z|X,Y)P(X)P(Y|X)\\ &=P(Y,Z|X)P(X) \end{align*} Which derives the third expression from the second.

(However I don't have any good intuition for what the third expression means. Does anyone else?)

$\endgroup$
0
16
$\begingroup$

We have

$$P(X,Y\mid Z) \tag1$$

Considering X and Y as a single event, we call them A. Now we have

$$P(A\mid Z) = P(X,Y\mid Z) \tag2$$

Using the Joint Probabilities Rule, we have

$$P(A,Z) = P(A\mid Z)\times P(Z) \tag3$$

So we can say that

$$P(A\mid Z) = \frac{P(A,Z)}{P(Z)} \tag4$$

We know that

$$P(A,Z) = P(Z,A) \tag5$$

Again using the Joint Probabilities Rule, we have

$$P(Z,A) = P(Z \mid A)\times P(A) \tag6$$

We defined $P(A)$ as the following

$$P(A) = P(X,Y) \tag7$$

Again using the Joint Probabilities Rule, we have

$$P(X,Y) = P(X\mid Y)\times P(Y) \tag8$$

Plugging $(8)$ into $(7)$, we have

$$P(A) = P(X\mid Y)\times P(Y) \tag9$$

Plugging $(9)$ into $(6)$, we have

$$P(Z,A) = P(Z\mid A)\times P(X\mid Y)\times P(Y) \tag{10}$$

Plugging $(10)$ into $(5)$ we have

$$P(A,Z) = P(Z\mid A)\times P(X\mid Y)\times P(Y) \tag{11}$$

Plugging $(11)$ into $(4)$, we have

$$P(A\mid Z) = \frac{P(Z\mid A)\times P(X\mid Y)\times P(Y)}{P(Z)} \tag{12}$$

Plugging $(12)$ into $(2)$, we have

$$P(X,Y\mid Z) = \frac{P(Z\mid A)\times P(X\mid Y)\times P(Y)}{P(Z)} \tag{13}$$

Observe that in $(13)$, using the Joint Probabilities Rule, we have

$$P(X,Y) = P(X\mid Y)\times P(Y) \tag{14}$$

Since we defined $P(A)$ is $P(X,Y)$, we have

$$P(A) = P(X\mid Y)\times P(Y) \tag{15}$$

Plugging $(15)$ into $(13)$, we have

$$P(X,Y\mid Z) = \frac{P(Z\mid A)\times P(A)}{P(Z)} \tag{16}$$

Observe that in $(16)$, using the Joint Probabilities Rule, we have

$$P(Z\mid A) = \frac{P(Z,A)}{P(A)} \tag{17}$$

Plugging $(17)$ into $(16)$, we have

$$P(X,Y\mid Z) = \frac{P(Z,A)}{P(Z)} \tag{18}$$

Now observe the following

$$P(Z,A) = P(Z,X,Y) = P(Y,Z,X) \tag{19}$$

Similar to what we did at the beginning, treating $Y$ and $Z$ as a single event and using the Joint Probabilities Rule, we have

$$P(Y,Z,X) = P(Y,Z\mid X)\times P(X) \tag{20}$$

Plugging $(20)$ into $(19)$, we have

$$P(Z,A) = P(Y,Z\mid X)\times P(X) \tag{21}$$

Plugging $(21)$ into $(18)$, we have

$$P(X,Y\mid Z) = \frac{P(Y,Z\mid X)\times P(X)}{P(Z)} \tag{22}$$

I don't know if this clarifies or complicates things more but nevertheless I wanted to include this here as well.

Right now, I can't prove why treating multiple joint events as if they were a single event is "legal".

$\endgroup$
6
  • $\begingroup$ Please edit your answer by adding $\rm\LaTeX$, for some basic information about writing math at this site see e.g. here, here, here and here. $\endgroup$ – Hakim Oct 26 '14 at 12:58
  • $\begingroup$ Can you explain why (5) P(A,Z) = P(Z,A). I don't understand why this is the same $\endgroup$ – Mr Bonjour Feb 5 '15 at 10:20
  • $\begingroup$ @MrBonjour Oh, actually I don't have a rigorous proof for that but it simply states that: "Probability of occurrence of A AND B together is the same as probability of occurrence of B AND A together.". As I said, I don't have a rigorous proof for it but both sides of the equation state: "The probability of occurrence of two (given) events at the same time.". Since they are occurring at the same time, I don't think that specifying an order to this has a meaning. So this might be a casual explanation for that. $\endgroup$ – Utku Feb 5 '15 at 10:54
  • $\begingroup$ Ok thank you. Actually I'm agree with your explanation. By the way your previous example give me a good grasp of the equation. Thank you again $\endgroup$ – Mr Bonjour Feb 6 '15 at 8:16
  • 1
    $\begingroup$ P(A,Z)=P(Z,A) because events Z and A are independent, not because they are simultaneous or not. If A and Z are not independent, then this equality is not true. Example of not independent events that occur simultaneously: A = encounter a person with long hair, Z = encounter a woman $\endgroup$ – Dmitry Avtonomov Apr 3 '18 at 11:43
1
$\begingroup$

It is easy to follow the following argumentation $$P(Z|X,Y)P(X,Y) = \frac{P(X,Y,Z)}{P(X,Y)}P(X,Y) = P(X,Y,Z)=\frac{P(X,Y,Z)}{P(X)}P(X)=P(Y,Z|X)P(X).$$

$\endgroup$
3
  • 11
    $\begingroup$ you are being unnecessarily tactless. we are here to help and get help, not to show off.. $\endgroup$ – Giovanni Sep 7 '15 at 6:19
  • $\begingroup$ I have to say that I found this explanation the easiest to follow, even so for the original, one-line version preceding the edit. Clearly, some folks might have taken offense with @royli prefacing his answer with "Isn't this obvious?" but for me, when explained that way, it was obvious! $\endgroup$ – Robert Apr 21 '19 at 14:14
  • $\begingroup$ I see this type of language all the time in lecture notes from the CS dept at my university... I think it's a stylistic trend. however, it drives me crazy. Oftentimes the claim is not "easy" or "trivial to see" or "obvious" in my opinion. $\endgroup$ – Ashish Jul 12 '20 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.