# Background:

Suppose we have two classes for p=2 variate observations, where the probability for class 1 follows MVN($$\mu_1$$, $$\Sigma$$) and the population for class 2 follows MVN($$\mu_2$$, $$\Sigma$$) where $$\mu_1 = \begin{pmatrix} 1 \\ 2 \end{pmatrix}$$, $$\mu_2 = \begin{pmatrix} 6 \\ 4 \end{pmatrix}$$, and $$\Sigma = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$$. Assume the prior probabilities for both are the same.

# Question:

Suppose we observe $$x = \begin{pmatrix} 3 \\ 5 \end{pmatrix}$$. Which class should x belong to given Bayes Classification Rule.

# My work:

Bayes Classification Rule: $$C_B(x) = \begin{cases} C_1 \space if\space \pi_1*f_1(x) > \pi_2*f_2(x) \\ C_2 \space if \space otherwise \end{cases}$$

Since our prior probabilities are equal and both populations have the same variance-covariance matrix I calculate the densities for both $$f_1$$ and $$f_2$$ where $$f_i = \frac{1}{2\pi\sqrt{3}}e^{\frac{-1}{2}*(x - \mu_i)^T\Sigma^{-1}(x - \mu_i)} \space i = 1,2$$

Then $$f_1 = 0.008910575$$ and $$f_2 = 0.001205915$$. Therefore x should go be classified as class 1 by the Bayes Classification Rule.

Is it normal to see values this small, or did I do something wrong here?

Thanks.

As far as I can tell, your approach is correct. I wouldn't worry about the values you get - take into account that the maximum value taken by $$f_i$$ is $$f_i(\mu_i) = \frac{1}{2\pi\sqrt{3}} \approx 0.0919$$ and that the normalized distances of the point $$x$$ from the means $$\mu_1$$ and $$\mu_2$$ are \begin{align} \sqrt{(x-\mu_1)^T \Sigma^{-1} (x-\mu_1)} &\approx 2.1602 & \sqrt{(x-\mu_2)^T \Sigma^{-1} (x-\mu_2)} &\approx 2.9439 \end{align} which means that we are "pretty far down the bell curve."
One extra suggestion I have for you, if I may, is that it's usually more convenient to work with the log-likelihood ratio and check its sign, instead of comparing the pdfs. In particular, in this case, let \begin{align} L(x) = \log \frac{f_1(x)}{f_2(x)} &= -\frac{1}{2} (x-\mu_1)^T \Sigma^{-1} (x-\mu_1) + \frac{1}{2} (x-\mu_2)^T \Sigma^{-1} (x-\mu_2) \\ &= (\mu_1 - \mu_2)^T \Sigma^{-1} x - \frac{1}{2} \mu_1^T \Sigma^{-1} \mu_1 + \frac{1}{2} \mu_2^T \Sigma^{-1} \mu_2. \end{align} Then, following your notation, $$C_B(x) = \begin{cases} C_1 &\text{if }L(x) > 0 \\ C_2 &\text{otherwise}. \end{cases}$$ Note that $$L(x)$$ is much simpler to compute than finding both $$f_1(x)$$ and $$f_2(x)$$, especially taking into account that the last two terms are constant and can be computed just once. In your case, you can easily check that $$L\biggl(\begin{pmatrix}3\\5\end{pmatrix}\biggr) = 2$$ and that it's indeed the same as taking $$\log(f_1(x)/f_2(x))$$ with the values you provide.