3
$\begingroup$

I'm implementing some stuff for machine learning and I ran across this post detailing some information on Bayes Theorem I was looking for:

https://stats.stackexchange.com/questions/31891/why-does-my-naive-bayes-classifier-only-give-me-probabilities-near-0

However, the reply from alto uses arg max. I see this alot, especially when dealing with Naive Bayes classification. Can anyone explain to me how it works? The wikipedia makes sense, except in this context I dont get why you would use this.

Thanks!

$\endgroup$
4
  • $\begingroup$ Usually you have probability distributions (which replace the various $P(\cdots)$), which you then minimise, either analytically or numerically. $\endgroup$
    – davin
    Commented Jul 29, 2014 at 14:59
  • $\begingroup$ By "how it works", are you asking about how the argmax function is implemented, or are you asking about the intuition behind argmax and why it's useful in this (Naive Bayes) context? $\endgroup$ Commented Jul 29, 2014 at 19:59
  • $\begingroup$ If it's the intuition you're seeking, then argmax is useful here because you want to find the class $C_i$ that a $D$ is most likely associated with. To do this, you compute probabilities for each $C_i$, and you choose the $C_i$ that maximizes this probability. Choosing such a $C_i$ is exactly what argmax does for you. $\endgroup$ Commented Jul 29, 2014 at 20:02
  • $\begingroup$ @VikramSaraph that makes it make much more sense. You're essentially doing the math and choosing the class that has the maximum probability. Would you mind going into how it's implemented for completeness? Also, add this as an answer so I can give you credit. $\endgroup$ Commented Jul 30, 2014 at 1:25

1 Answer 1

2
$\begingroup$

In Naive Bayes classification, argmax is useful because you want to find the class $C_i$ that an object $D$ is most likely associated with. To do this, you compute a probability for $D$ belonging to each $C_i$, and you choose the $C_i$ that maximizes this probability. Choosing such a $C_i$ is exactly what argmax does for you.

I didn't mean to make computing argmax sound more complex than it actually is, but I'll talk about it since you asked. In Naive Bayes, there are usually few enough classes to compute argmax using brute force: simply compute the probability for all $C_i$ and return the $C_i$ that results in largest one. In general (that is, outside of Naive Bayes), computing the argmax of a function is as hard as maximizing it, so for some functions, using a brute force approach to compute their argmax would take an eternity.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .