# Using LDA decision boundary inequality to classify an observation X

I have a single regressor $$X$$ and response $$y$$, where $$y=n_k/n$$ if $$X$$ is of class $$k$$, and $$k=1,2$$. Let $$n_k$$ denote the number of observations in class $$k$$, and $$n$$ the total number of observations. Denote $$\mu_k$$ as the mean of class $$k$$, $$\Sigma$$ the covariance matrix of X, and $$\pi_k$$ the probability of class $$k$$ occurring. I have the general discriminant of $$\delta(x)=x'\Sigma^{-1}\mu_k-\frac{1}{2}\mu_k'\Sigma^{-1}\mu_k+\log(\pi_k)$$,

and I know the decision boundary between classes 1 and 2 is given by $$\log(n_2/n)-\frac{1}{2}\mu_2'\Sigma^{-1}\mu_2+x'\Sigma^{-1}\mu_2=\log(n_1/n)-\frac{1}{2}\mu_1'\Sigma^{-1}\mu_1+x'\Sigma^{-1}\mu_1$$

I let $$\hat\sigma^2$$ estimate $$\Sigma$$ and $$\hat\mu_k$$ estimate $$\mu_k$$, and then by treating these as scalars and using algebraic manipulation, I obtain

$$\frac{(\hat\mu_2-\hat\mu_1)x}{\hat\sigma^2}=\frac{(\hat\mu^2_2-\hat\mu^2_1)}{2\hat\sigma^2}-\log(n_2/n_1)$$

For any $$X$$ that doesn't satisfy this equation, I suspect that determining which side of this equation is greater for a given $$X$$ will tell me which class $$X$$ belongs to, but I can't figure out which side would correspond to which class (if this is a correct assumption).

## 1 Answer

When we are not sure which side does a classifier assign a data point to, a possible way is to substitute some points that we know it belongs to a particular class to observe the outcome. We can try out with $$\hat{\mu}_1$$ and $$\hat{\mu}_2$$ where $$\hat{\mu}_1\ne\hat{\mu}_2$$

The boundary is

$$\frac{(\hat\mu_2-\hat\mu_1)x}{\hat\sigma^2}-\frac{(\hat\mu^2_2-\hat\mu^2_1)}{2\hat\sigma^2}=-\log(n_2/n_1)$$

Let $$f(x) = \frac{(\hat\mu_2-\hat\mu_1)x}{\hat\sigma^2}-\frac{(\hat\mu^2_2-\hat\mu^2_1)}{2\hat\sigma^2}=\frac{(\hat{\mu}_2-\hat{\mu}_1)}{\hat{\sigma}^2}\left( x-\frac{\hat{\mu}_1 + \hat{\mu}_2}2\right).$$

$$f(\hat{\mu}_1) =\frac{(\hat{\mu}_2-\hat{\mu}_1)}{\hat{\sigma}^2}\left( \hat{\mu}_1-\frac{\hat{\mu}_1 + \hat{\mu}_2}2\right)=\frac{-(\hat{\mu}_2 - \hat{\mu}_1)^2}{2\hat{\sigma}^2} <0$$

$$f(\hat{\mu}_2) =\frac{(\hat{\mu}_2-\hat{\mu}_1)}{\hat{\sigma}^2}\left( \hat{\mu}_2-\frac{\hat{\mu}_1 + \hat{\mu}_2}2\right)=\frac{(\hat{\mu}_2 - \hat{\mu}_1)^2}{2\hat{\sigma}^2} >0$$

Hence if $$f(x) < -\log(n_2/n_1)$$, it belongs to the first class.

If $$f(x) > -\log(n_2/n_1)$$, it belongs to the second class.