1
$\begingroup$

Is there some relationship between the correlation of two random variables, and Bayes Theorem?

A bit of background intuition,

if W = random variable denoting number of women in a room, and L = random variable denoting number of long-haired people in the same room, we can infer about one variable given the other either using the correlation value or the conditional expectation value as given by Bayes Theorem (though Bayes deals with events, probability densities are tied to expectations anyway)

Thanks

$\endgroup$

2 Answers 2

1
$\begingroup$

There is no essential difference between Bayes' Theorem and the formula $P(A|B)\cdot P(B)=P(AB)$. Each can easily be used to prove the other.

$\endgroup$
3
  • $\begingroup$ what? by correlation I meant cov(W,L)/(std(W)*std(L)) $\endgroup$
    – ejang
    Feb 17, 2013 at 23:28
  • $\begingroup$ ahhhhh, in that case, what kind of relation can you expect between a theorem and a number???? $\endgroup$ Feb 17, 2013 at 23:29
  • $\begingroup$ sorry, i'll rephrase my question above $\endgroup$
    – ejang
    Feb 17, 2013 at 23:32
0
$\begingroup$

The relationship is not so much with Bayes' Theorem as it is with inference and therefore conditional probabilities.

Intuitively, if two variables $X$ and $Y$ are uncorrelated then you wouldn't expect to be able to infer anything much about $Y$ from a realization of $X$.

Here is a simple example of 2 realisations of random variables that are uncorrelated (you might call this the training set of the classifier):

X  Y
----
1  1
1 -1

It is simple to calculate that the variance is zero: $\mathbb{E}(XY) = 0$.

By definition and since we only have $1$s and $-1$s: $$\begin{align} \mathbb{E}(XY) = & \, \mathbb{P}(X=1, Y=1) + \mathbb{P}(X=-1, Y=-1) \\ &- \mathbb{P}(X=-1, Y=1) - \mathbb{P}(X=1, Y=-1) \end{align} $$

If we use the definition $\mathbb{P}(Y|X) = \mathbb{P}(X, Y) / \mathbb{P}(X)$ throughout we can refactor as:

$$\begin{align} \mathbb{E}(XY) = & \mathbb{P}(X=1) \, \big[\mathbb{P}(Y=1|X=1) - \mathbb{P}(Y=-1|X=1)\big] \, + \\ &\mathbb{P}(X=-1) \, \big[\mathbb{P}(Y=-1|X=-1) - \mathbb{P}(Y=1|X=-1)\big] \end{align} $$

In my simple training set $X$ is never $-1$ and so I can reduce it further: $$ \mathbb{E}(XY) = \big[\mathbb{P}(Y=1|X=1) - \mathbb{P}(Y=-1|X=1)\big] $$ This equality explains how the zero correlation is related to the fact that $X$ is not a good signal for the classification of $Y$: the conditional probabilities on the right are equal so we cannot classify $Y$ as more likely to be a $1$ or a $-1$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .