121
$\begingroup$

What is the difference between regression and classification, when we try to generate output for a training data set $x$?

$\endgroup$

10 Answers 10

139
$\begingroup$

Regression: the output variable takes continuous values.

Classification: the output variable takes class labels.

$\endgroup$
  • 1
    $\begingroup$ The output variable (I assume you mean response variable) can also be boolean with logistic regression right? $\endgroup$ – Justin Bozonier Jul 24 '13 at 2:06
  • 3
    $\begingroup$ Binary logistic regression also outputs a continuous variable. Specifically, the regression estimates the odds that a variable is in a given class as a function of the predictor variables. $\endgroup$ – James Thompson Jul 29 '13 at 16:45
  • $\begingroup$ Binary Logistic regression produces a continuous output but not to try to give a continuous output at the data (regression) but in order to classify them in two classes $\endgroup$ – K. Stasko Mar 8 '14 at 13:05
  • $\begingroup$ +1. Not so often that I see such a clear answer! It makes the question look so simple & easy. $\endgroup$ – 0xc0de Feb 21 '16 at 6:53
  • 3
    $\begingroup$ This is a much simpler answer, thanks. If understood right - Let's say that our subject is the weather tomorrow. Regression - Predict the temperature Classification - Will it be hot or not? $\endgroup$ – Yehonatan Aug 9 '16 at 18:43
111
$\begingroup$

Regression involves estimating or predicting a response.

Classification is identifying group membership.

$\endgroup$
  • 19
    $\begingroup$ signed up to up vote this response. i love the plain English description. $\endgroup$ – Julian Apr 26 '13 at 5:49
  • $\begingroup$ Rigorously speaking, "estimating a response" is a incorrect terminology use. $\endgroup$ – Zhanxiong Jan 12 '18 at 6:01
  • $\begingroup$ @Zhanxiong Well what's the correct terminology then? $\endgroup$ – endolith Jan 23 '18 at 16:08
  • 2
    $\begingroup$ @endolith In a narrow sense, we say "estimate parameter" but "predict/forecast response". $\endgroup$ – Zhanxiong Jan 24 '18 at 3:27
25
$\begingroup$

Regression and classification are both related to prediction, where regression predicts a value from a continuous set, whereas classification predicts the 'belonging' to the class.

For example, the price of a house depending on the 'size' (in some unit) and say 'location' of the house, can be some 'numerical value' (which can be continuous): this relates to regression.

Similarly, the prediction of price can be in words, viz., 'very costly', 'costly', 'affordable', 'cheap', and 'very cheap': this relates to classification.

Each class may correspond to some range of values.

$\endgroup$
  • 2
    $\begingroup$ I joined the Math community just to upvote this $\endgroup$ – Marvin Thobejane Jan 29 at 18:56
15
$\begingroup$

Given the following

$$f: x \rightarrow y$$

If $y$ is discrete/categorical variable, then this is classification problem.

If $y$ is real number/continuous, then this is a regression problem.

$\endgroup$
  • 6
    $\begingroup$ This seems to simply reiterate answers from long ago. If you have something new to add, please clarify your answer. $\endgroup$ – robjohn Jul 27 '14 at 12:51
10
$\begingroup$

Regression: given a set of data, find the best relationship that represents the set of data.

Classification: given a known relationship, identify the class that the data belongs to.

We can see that regression and classification start from opposing ends: to find a pattern or to find the pattern that it belongs to.

$\endgroup$
8
$\begingroup$

Regression and classification can work on some common problems where the response variable is respectively continuous and ordinal.

But the result is what would make us choose between the two.

For example, simple/hard classifiers (e.g. SVM) simply try to put the example in specific class (e.g. whether the project is "profitable" or "not profitable", and doesn't account for how much), where regression can give an exact profit value as some continuous value.

However, in the case of classification, we can consider probabilistic models (e.g. logistic regression), where each class (or label) has some probability, which can be weighted by the cost associated with each label (or class), and thus give us with a final value on basis of which we can decide to put it some label or not. For instance, label $A$ has a probability of $0.3$, but the payoff is huge (e.g. 1000). However, label $B$ has probability $0.7$, but the payoff is very low (e.g. $10$). So, for maximizing the profit, we might label the example as label $A$ instead of $B$.

Note: I am still not an expert, perhaps someone would rectify if I am wrong in some part.

$\endgroup$
4
$\begingroup$

http://www.differencebetween.com/difference-between-classification-and-vs-regression/

Classification trees have dependent variables that are categorical and unordered.

Regression trees have dependent variables that are continuous values or ordered whole values.

$\endgroup$
3
$\begingroup$

Regression means to predict the output value using training data.

Classification means to group the output into a class.

For example, we use regression to predict the house price (a real value) from training data and we can use classification to predict the type of tumor (e.g. "benign" or "malign") using training data.

$\endgroup$
  • 1
    $\begingroup$ It's funny when you read this answer while referring to a machine learning course at Coursera ;) . $\endgroup$ – 0xc0de Feb 21 '16 at 6:58
2
$\begingroup$

Regression is defined as $E[Y \mid X]$ (i.e. the expectation of $Y$ given $X$).

A subset of these types of models where $Y$ is binary or categorical, including logistic regression and multinomial regression along with many machine learning algorithms that essentially have the same target (such as classification trees), are useful for classification.

$\endgroup$
1
$\begingroup$

It is important to be clear when using terms like regression, classification and prediction to discriminate between the task you are performing and the method used to perform it. A classification task involves taking an input and labelling it as belonging to a given class, so the output is categorical. On the other hand, a prediction task involves predicting a continuous valued output.

Methods for achieving these tasks include regression, in which a continuous valued output is estimated (or, rather, the expected value of a distribution on a continuous variable is estimated, conditional on a given set of input values). This can be used to carry out a prediction task, as you would expect. It can also be used to carry out a classification task, for example using logistic regression to estimate the log odds of the input pattern belonging to a given class. In this case, the task is classification, the method is regression.

Classification methods simply generate a class label rather than estimating a distribution parameter. K nearest neighbour is a good example where the task and the method are both called classification.

$\endgroup$

protected by Community Aug 1 '18 at 22:55

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.