Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Could someone provide differences between these two techniques? Also, I would very much appreciate if you could give an example of a ML method that falls into discriminative model or gnerative one. For instance, is Perceptron discriminative? K-Means?

share|improve this question

2 Answers 2

up vote 2 down vote accepted

Let $X$ be your observed data and let $Y$ be their unobserved/hidden properties. In a ML setting, $Y$ usually holds the categories of $X$.

A generative model models their joint distribution, $P(X,Y)$.

A discriminative model models the posterior probability of the categories, $P(Y|X)$.

Depending on what you want to do, you choose between generative versus discriminative modeling. For example, if you are interested in doing classification, a discriminative model would be your choice because you are interested in $P(Y|X)$. However, you can use a generative model, $P(X,Y)$, for classification, too.

A generative model allows you to generate synthetic data ($X$) using the joint but you cannot do this with a discriminative model.

share|improve this answer

Consider a classification problem at the simplest approach one can get to its goal by designing a discriminant function. for example: a perceptron $y=sign(\theta^Tx)$ you can learn the parameters by a algorithm. in this case you should use Perceptron Learning Rule.

the above approach is not probabilistic and is called 'discriminant function' approach.

a better goal is trying to predict the labels probabilistically. the best goal is to have P(y|X) that is called the posterior.

Example of a discriminative approach: logistic regression

$P(y=1|X)=g(\theta^Tx)$,where g is the sigmoid function.

Example of a generative model:

$P(y|X)=P(X|y)P(y)$

you can model $P(X|y=1)=N(\mu_1,\Sigma_1)$ and $P(X|y=0)=N(\mu_0,\Sigma_0)$ and estimates its parameters from data. (the simple approach is Maximum likelihood).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.