For conditionally independent features $f_i$, Naive Bayes Classification gives me the classifier

$Classifier(f) := \arg \max_{k} P(C=k) · ∏^n_{i=1} P(f_i|C=k)$

for classes $k$. I understand that for Gaussian Naive Bayes, I can assume normally distributed features, yielding

$Classifier(f) := \arg \max_k P(C=k) · ∏^n_{i=1} \frac{1}{\sqrt{(2πσ_{k,i})}} e^{( -\frac{(f_i - μ_{k,i})^2}{2σ_{k,i})}}$

where $μ_{k,i}$ is the mean of class $k$ and feature $f_i$ (and similiar for variance $σ_{k,i}$).

But where is the "learning step" in this whole procedure?


1 Answer 1


Naive Bayes uses the class labels from the training set to learn $P(C=k)$ and $P(f_i|C=k)$ directly from the training data.

You learn the mean and variance of each class conditional distribution for each feature just as you would the mean and variance for any other problem. You treat the $f_i$ of each class as if they were their own independent sample.

  • $\begingroup$ thanks for your answer! What I still don't get is how can we calculate the mean of a class and every feature? I understand how to calculate the mean of many features of a class, but of a single feature?? $\endgroup$
    – MJimitater
    Commented Feb 25, 2020 at 12:21
  • $\begingroup$ You’re calculating the mean of each feature within that class. You’re treating the individual features as independent so don’t worry about jointly estimating the distribution of features. $\endgroup$ Commented Feb 28, 2020 at 20:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .