For conditionally independent features $f_i$, Naive Bayes Classification gives me the classifier

$Classifier(f) := \arg \max_{k} P(C=k) · ∏^n_{i=1} P(f_i|C=k)$

for classes $k$. I understand that for Gaussian Naive Bayes, I can assume normally distributed features, yielding

$Classifier(f) := \arg \max_k P(C=k) · ∏^n_{i=1} \frac{1}{\sqrt{(2πσ_{k,i})}} e^{( -\frac{(f_i - μ_{k,i})^2}{2σ_{k,i})}}$

where $μ_{k,i}$ is the mean of class $k$ and feature $f_i$ (and similiar for variance $σ_{k,i}$).

But where is the "learning step" in this whole procedure?


Naive Bayes uses the class labels from the training set to learn $P(C=k)$ and $P(f_i|C=k)$ directly from the training data.

You learn the mean and variance of each class conditional distribution for each feature just as you would the mean and variance for any other problem. You treat the $f_i$ of each class as if they were their own independent sample.

  • $\begingroup$ thanks for your answer! What I still don't get is how can we calculate the mean of a class and every feature? I understand how to calculate the mean of many features of a class, but of a single feature?? $\endgroup$ – MJimitater Feb 25 '20 at 12:21
  • $\begingroup$ You’re calculating the mean of each feature within that class. You’re treating the individual features as independent so don’t worry about jointly estimating the distribution of features. $\endgroup$ – MentatOfDune Feb 28 '20 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.