# Second-order Bayesian inference

We have three events $A$, $B$ and $C$ in question, and given appropriate priors, we derive the posterior $\Pr(A|B)$. Now we want to derive a 'second-order' posterior $\Pr(A|B,C)$ by using the 'first-order' posterior $\Pr(A|B)$ as the prior.

First of all, does this mean that $\Pr(A|B,C)$ is the same as $\Pr((A|B)|C)$? If so, is the following correct:

$$\Pr(A|B,C)=\frac{\Pr(C|(A|B))\cdot\Pr(A|B)}{\Pr(C)}$$

and how do we derive $\Pr(C|(A|B))$?

-
$A|B$ is not an event, and conditioning on $A|B$ makes no sense. What is true is that $$P(A|B,C)=\frac{P(A,B,C)}{P(B,C)}=\frac{P(C|A,B)P(A,B)}{P(C|B)P(B)}=\frac{P(C|A‌​,B)}{P(C|B)}P(A|B)$$ –  Dilip Sarwate Feb 29 '12 at 13:49
@DilipSarwate, and what is $\Pr(C|A,B)$? –  Aaron Feb 29 '12 at 13:59
$\Pr(C|A,B)$ means the probability that the event $C$ occurs, given that the two events $A$ and $B$ both occur. –  Henry Feb 29 '12 at 14:11
@Henry, I understand the meaning of it, but how do I derive it mathematically? I currently know $\Pr(A|B)$, $\Pr(C|B)$, $\Pr(B|A)$, $\Pr(C|A)$ and $\Pr(A)$. –  Aaron Feb 29 '12 at 14:21

If what you're interested in is the probability of $A$ given $B$ and $C$ occur, this is just $\Pr(A|B \wedge C)$ (Or $\Pr(A|B\cap C)$ if you prefer that way of talking).

As was mentioned in the comments, $A|B$ isn't an event in the algebra of events, so it's meaningless to say $\Pr((A|B)|C)$.

So, using the definition of conditionalisation (And using $XY$ to mean $X\wedge Y$):

$$\Pr(A|BC) = \frac{\Pr(ABC)}{\Pr(BC)}$$

Using the fact that $\Pr(XY)=\Pr(X|Y)\Pr(Y)$ twice, we find:

$$\Pr(A|BC) = \frac{\Pr(AB|C)\Pr(C)}{\Pr(B|C)\Pr(C)} = \frac{\Pr(AB|C)}{\Pr(B|C)}$$

Depending on what probabilities are known, many other manipulations are possible. See the wikipedia page on Bayes' theorem

-
so how would you derive $\Pr(A|B \wedge C)$ given the information above? –  Aaron Feb 29 '12 at 15:42
@Aaron Given what information? Which conditional probabilities are known? –  Seamus Feb 29 '12 at 15:49
as mentioned in my comment to Henry above, we know $\Pr(A|B)$, $\Pr(C|B)$, $\Pr(B|A)$, $\Pr(C|A)$, and $\Pr(A)$. –  Aaron Feb 29 '12 at 16:04
Well that is just a matter of algebraic manipulation. If you can't work it out, but it might not be possible. You might not have enough information to determine it. Also I suggest you add that information to your question. –  Seamus Feb 29 '12 at 17:17
@Aaron a minute's quick messing with the numbers suggests to me that you don't have enough information to solve it. If you knew $\Pr(C|AB)$ you could solve it using the formula on the wiki page I linked to, since you know all the other elements apart from $\Pr(B)$ and you can derive that from what you know. –  Seamus Feb 29 '12 at 17:37

From what you know:

Pr(A|B), Pr(C|B), Pr(B|A), Pr(C|A) and Pr(A)

You can derive the joint distribution from the factors: P(A, B, C) = P(C | B) P(B | A) P(A). This set of equations indicates that the structure of your conditional independence assumptions is as follows:

A -> B -> C

Which means that C is conditionally independent of of A given B (once we know the value of B, C provides no additional information about A). In this case P(A | B, C) = P(A | B).

-