# Conditional probability - a formal discussion

This is a rather philosophical question.

P[B|A] is formally defined as P[B and A]/P[B] where A and B are events in a sigma algebra and P is a probability mass function.

That is, P[B|A] is just a division of two numbers. If so, how come there are problems where we find it hard to calculate P[B and A] as well as P[B], but it is easy for us to reason about P[B|A] and so we assign a value to P[B|A] immediately without going through the division? (I can't think of an example for this, but I surely recall there are such cases. Can anyone share an example?)

To be more concrete, I'd be happy to see an example where it's hard\impossible to calculate P[A and B] or P[B] but it is easy to reason about P[A|B] on intuitive levels along with a justification for this reasoning (I'm talking about sample space and probability function definitions).

-
I'm not sure the definition you quote is theoretically satisfying. For example, suppose $X$ and $Y$ are random variables each taking values on $[0,1]$, with a continuous joint distribution. Then it ought to be meaningful to speak of $P(X>\frac12\mid Y=\frac13)$, even though $P(Y=\frac13)$ is $0$... – Henning Makholm Nov 20 '11 at 22:45
@Henning, your quibble is unfounded. If the random variable $Y$ is absolutely continuous ($P(Y=y)=0$ for every $y$), then the function $g:y\mapsto P(X\gt\frac12\mid Y=y)$ is defined up to a null set hence $g(\frac13)$ has no meaning on its own. – Did Nov 20 '11 at 23:11
@Didier, by "unfounded" do you mean that there's a solution I haven't thought of, or that I shouldn't ought to wish for a solution at all? – Henning Makholm Nov 20 '11 at 23:14
Let's stick to the discrete case and assume P[B] > 0 – Leo Nov 20 '11 at 23:21
@Henning, I think my first comment is clear enough but let me try again: you write Then it ought to be meaningful to speak of $P(X>1/2∣Y=1/3)*, even though$P(Y=1/3)$is$0$... As I said, it is not, and since this seems to be the justification of your quibble, I do not understand said quibble. – Did Nov 21 '11 at 20:15 ## 5 Answers I think you mean$P(A|B)$rather than$P(B|A)$; I'll assume that. It might happen that event$B$, if it happens, controls the conditions for$A$to happen, which does not imply that one has any idea of how probable$B$is. As an extreme case,$B$might logically imply$A$, in which case$P(A|B)=1$regardless. Another example is if someone tosses a coin but I have no idea whether the coin is fair; for the events$A$: I win the toss, and$B$: the coin is fair, I know by definition that$P(A|B)=0.5$, even though I know nothing about$P(B)$or$P(A\cap B)$. - Can you elaborate on the "by definition that P(A|B)=0.5" part? What I'm looking for is a rigorous explanation instead of the intuitive step. – Leo Nov 22 '11 at 22:33 My description assumed that the events include selecting a coin and tossing it. Now for any single coin, the coin$c$is considered fair, by definition, if among those events$B_c$where this coin is selected the outcome "heads" is equally probable as "tails". That means$P(B_c\cap A)=P(B_c\cap\lnot A)=0.5P(B_c)$. Now uniting over all such fair coins to get$B$gives$P(A\cap B)=0.5P(B)$and therefore$P(A|B)=0.5$. – Marc van Leeuwen Nov 26 '11 at 13:46 Take the following setup: Box I contains one red marble and one green marble. Box II contains one red marble and two green marbles. Someone is about to toss a coin. If it comes up heads, the person will draw one marble at random from box I. If it comes up tails, the person will draw one marble at random from box II. Informally, it seems clear that if the coin comes up heads, the chance the marble will be red is 1/2. But the formal definition of conditional probability can't be applied at this point. You need a sample space, say: HR, HG, TR, TG; and an assignment of non-negative numbers to each of the four elements of the sample space. (The numbers should add to 1). How about assigning 1/4 to each? Note that this is a perfectly legitimate sample space. So you can use the formal definition and you will get P(Red|Heads) = 1/2 . (Heads: {HR, HG} and Red: {HR, TR}). However, you will also get P(Red|Tails) = 1/2. That does not match the informal chance of 1/3. Again, though, the sample space is legitimate. How could you get four numbers so that the formal definition would lead to answers that match the informal chances? Use the informal rule:  chance of . . . followed by _ _ _ is equal to conditional chance of _ _ _ given . . . times chance of . . .  So to define the usual sample space in this setup, you need to be able to recognize (informal) conditional chances and multiply appropriately. (The statement of Bayes rule in texts is often misleading in this respect.) Roger Purves - I'll give you an example where it is potentially hard to find the values of two unknowns$x$and$y$explicitly but where it is easy to talk about$\frac{x}{y}$: Let$x$be the number of hydrogen atoms in a given and unknown quantity of water and$y$be the number of oxygen atoms. Now it might be really difficult to measure or just fathom the number of hydrogen or oxygen atoms in a large (but not precisely known) quantity of water, whereas we know for sure that$\frac{x}{y}=2$. Put differently, the quotient of two numbers stores in general only a fraction of the data contained in the sole numbers. - The proportion of voters in the state who will vote "yes" in next week's referendum is$R$. Say you decide to represent your uncertainty about the value of$R$by saying$R$has a certain probability distribution supported on the interval$[0,1]$. 1000 voters will be randomly chosen and asked whether they will vote "yes". The number$X$who answer "yes" is a random variable whose conditional probability distribution given$R$is a binomial distribution, so we can say $$\Pr(X = x\mid R) = \binom{1000}{x}R^x (1-R)^{1000-x}.$$ After that, we can find$\Pr(X=x)\$ by saying that it's equal to the expected value of the expression above and finding that.

That's one example.

-

Role of Conditional Probability - My thoughts:

A need for representing event in the presence of prior knowledge: Consider the probability of drawing a king of heart randomly from a standard deck of 52 cards. The probability of this event without any prior knowledge is 1/52. However, if one learns that the card drawn is red, then the probability of getting a king of heart becomes 1/26. Similarly, if one gathers knowledge that the card drawn is a face card (ace, king, queen or jack) then the probability gets shifted to 1/16. So we see that representing an event in the presence of prior knowledge is important and conditional event representation of (A|H) is the most adopted representation to solve this need. No matter what representation we adopt, we can agree that conditional event solves an unique concern of representing conditional knowledge.

What is most elemental - unconditional Vs. conditional: The debate whether conditional probability is more elemental than (unconditional) probability remains as an enticing subject for many statistician [1] as well as philosophers [2]. While the most adopted notation of conditional probability and its ratio representation viz. P(A|H)=P(AH)/P(H) where P(H)>0 indicates (unconditional) probability is more elemental; the other school of thoughts has their logic too. For, them, when we say probability of getting face value of 2 in a random throw of a fair dice is 1/6, we apply prior knowledge that all throw will lands perfectly on a face such that a face will be visible unambiguously, or that the dice will not break into pieces when rolled and so on. Therefore we apply a prior knowledge in order to determine a sample space of six face values. No matter what kind of probability is the most elemental, following the notation of conditional probability, we can agree that we speak of (unconditional) probability when we’ve accepted a sample space as the super most population and we’re not willing to get astray by adding further sample points to this space. Similarly, we speak of conditional probability when we focus on an event with respect to the sub-population of the super-most (absolute in this sense) population.

Is there any case which can be solved only by conditional probability: Once again, as long as we accept the ratio representation of the conditional probability, we see that conditional probability can be expressed in terms of unconditional probability. Thus, conceptually, any problem where conditional probability is used, can also be solved without use of conditional probability as well. However, we must appreciate that for cases where population and sub-population are not part of the same experiment, the use of conditional probability is really useful (not necessarily inevitable). To explain this further, in case of finding probability of a king of heart given that the card is red, we don’t really need conditional probability because the population of 52 cards and sub-population of 26 red cards are very clear to us. However, for cases such as applying a medicinal test on a cow to determine if it has mad-cow-disease, if we know false positive and false negative probabilities of the test, then to find out probability that a cow has disease given that it has tested positive, conditional probability can be used with great effect. If I may bring an analogy of ‘plus’ and ‘multiplication’ symbols of mathematics, we all know that any problem that uses multiplication symbol, can also be solved without it by mere use of ‘plus’ symbol. Similarly, in terms of solving problems, conditional probability can be avoided altogether just like multiplication symbol in mathematics. Still, we can appreciate the usefulness of conditional probability just like we can appreciate the use of multiplication in mathematics.

----------Bibliography------------

[1] H. Nguyen and C. Walker, “A history and introduction to the algebra of conditional events and probability logic,” Systems, Man and Cybernetics, IEEE Transactions on
(Volume:24 , Issue: 12 ), pp. 1671 - 1675, Dec 1994.

[2] A. Hájek, “Conditional Probability,” Handbook of the Philosophy of Science. Volume 7: Philosophy of Statistics., vol. 7, p. 99, 2011.

-
Please define "conditional event" and "conditional event representation". These seem bad terminology. – Did Aug 5 '13 at 8:52
Conditional Event: An event (A) when expressed under the knowledge of another event (H) is called a conditional event expressed as (A|H) <br/> Conditional Event Representation: Representing conditional events by using more formal symbols rather than using natural languages like English. – BuckCherry Aug 5 '13 at 9:06
What does "event when expressed under the knowledge of another event" mean? By the way, did you coin these terms yourself? There is some pre-existing body of knowledge in the field, you know... – Did Aug 5 '13 at 10:03
@Did - I know that there exist quite a body of knowledge in this field and I must confess that I'm no expert in that. Having said that, I guess it would be appropriate to say the definition of the term 'conditional event' was mine even though I don’t think I coined the term myself. I rather believe I picked it from various papers (see Bibliography). Anyway, it seems you're knowledgeable in this field and that you're trying to say something. If so, please, enlighten me/us by all means. I would rather appreciate your candid (and hopefully elaborate) feedback, rather than some hinting comments. – BuckCherry Aug 5 '13 at 10:29
Naively, I feel that I am saying things rather than trying to say them.... Anyway, re the question, my advice is to stick to @Marc's answer. – Did Aug 7 '13 at 23:30