2
$\begingroup$

I'm implementing some stuff for machine learning and I ran across this post detailing some information on Bayes Theorem I was looking for:

https://stats.stackexchange.com/questions/31891/why-does-my-naive-bayes-classifier-only-give-me-probabilities-near-0

However, the reply from alto uses arg max. I see this alot, especially when dealing with Naive Bayes classification. Can anyone explain to me how it works? The wikipedia makes sense, except in this context I dont get why you would use this.

Thanks!

$\endgroup$
  • $\begingroup$ Usually you have probability distributions (which replace the various $P(\cdots)$), which you then minimise, either analytically or numerically. $\endgroup$ – davin Jul 29 '14 at 14:59
  • $\begingroup$ By "how it works", are you asking about how the argmax function is implemented, or are you asking about the intuition behind argmax and why it's useful in this (Naive Bayes) context? $\endgroup$ – Vikram Saraph Jul 29 '14 at 19:59
  • $\begingroup$ If it's the intuition you're seeking, then argmax is useful here because you want to find the class $C_i$ that a $D$ is most likely associated with. To do this, you compute probabilities for each $C_i$, and you choose the $C_i$ that maximizes this probability. Choosing such a $C_i$ is exactly what argmax does for you. $\endgroup$ – Vikram Saraph Jul 29 '14 at 20:02
  • $\begingroup$ @VikramSaraph that makes it make much more sense. You're essentially doing the math and choosing the class that has the maximum probability. Would you mind going into how it's implemented for completeness? Also, add this as an answer so I can give you credit. $\endgroup$ – user3389436 Jul 30 '14 at 1:25
1
$\begingroup$

In Naive Bayes classification, argmax is useful because you want to find the class $C_i$ that an object $D$ is most likely associated with. To do this, you compute a probability for $D$ belonging to each $C_i$, and you choose the $C_i$ that maximizes this probability. Choosing such a $C_i$ is exactly what argmax does for you.

I didn't mean to make computing argmax sound more complex than it actually is, but I'll talk about it since you asked. In Naive Bayes, there are usually few enough classes to compute argmax using brute force: simply compute the probability for all $C_i$ and return the $C_i$ that results in largest one. In general (that is, outside of Naive Bayes), computing the argmax of a function is as hard as maximizing it, so for some functions, using a brute force approach to compute their argmax would take an eternity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.