What is the difference between regression and classification, when we try to generate output for a training data set $x$?
$\begingroup$
$\endgroup$
Add a comment
|
11 Answers
$\begingroup$
$\endgroup$
6
Regression: the output variable takes continuous values.
Classification: the output variable takes class labels.
-
1$\begingroup$ The output variable (I assume you mean response variable) can also be boolean with logistic regression right? $\endgroup$ Commented Jul 24, 2013 at 2:06
-
5$\begingroup$ Binary logistic regression also outputs a continuous variable. Specifically, the regression estimates the odds that a variable is in a given class as a function of the predictor variables. $\endgroup$ Commented Jul 29, 2013 at 16:45
-
$\begingroup$ Binary Logistic regression produces a continuous output but not to try to give a continuous output at the data (regression) but in order to classify them in two classes $\endgroup$ Commented Mar 8, 2014 at 13:05
-
$\begingroup$ +1. Not so often that I see such a clear answer! It makes the question look so simple & easy. $\endgroup$– 0xc0deCommented Feb 21, 2016 at 6:53
-
3$\begingroup$ This is a much simpler answer, thanks. If understood right - Let's say that our subject is the weather tomorrow. Regression - Predict the temperature Classification - Will it be hot or not? $\endgroup$ Commented Aug 9, 2016 at 18:43
$\begingroup$
$\endgroup$
5
Regression involves estimating or predicting a response.
Classification is identifying group membership.
answered Aug 15, 2012 at 20:54
-
21$\begingroup$ signed up to up vote this response. i love the plain English description. $\endgroup$– JulianCommented Apr 26, 2013 at 5:49
-
$\begingroup$ Rigorously speaking, "estimating a response" is a incorrect terminology use. $\endgroup$ Commented Jan 12, 2018 at 6:01
-
$\begingroup$ @Zhanxiong Well what's the correct terminology then? $\endgroup$– endolithCommented Jan 23, 2018 at 16:08
-
2$\begingroup$ @endolith In a narrow sense, we say "estimate parameter" but "predict/forecast response". $\endgroup$ Commented Jan 24, 2018 at 3:27
-
$\begingroup$ I was referring to estimating the expectation of the output variable given the regressors rather than estimating the parameters of the regressors. In regression both can be done. $\endgroup$ Commented Jul 3, 2019 at 16:50
$\begingroup$
$\endgroup$
1
Regression and classification are both related to prediction, where regression predicts a value from a continuous set, whereas classification predicts the 'belonging' to the class.
For example, the price of a house depending on the 'size' (in some unit) and say 'location' of the house, can be some 'numerical value' (which can be continuous): this relates to regression.
Similarly, the prediction of price can be in words, viz., 'very costly', 'costly', 'affordable', 'cheap', and 'very cheap': this relates to classification.
Each class may correspond to some range of values.
-
2$\begingroup$ I joined the Math community just to upvote this $\endgroup$ Commented Jan 29, 2019 at 18:56
$\begingroup$
$\endgroup$
1
Given the following
$$f: x \rightarrow y$$
If $y$ is discrete/categorical variable, then this is classification problem.
If $y$ is real number/continuous, then this is a regression problem.
-
6$\begingroup$ This seems to simply reiterate answers from long ago. If you have something new to add, please clarify your answer. $\endgroup$– robjohn ♦Commented Jul 27, 2014 at 12:51
$\begingroup$
$\endgroup$
Regression: given a set of data, find the best relationship that represents the set of data.
Classification: given a known relationship, identify the class that the data belongs to.
We can see that regression and classification start from opposing ends: to find a pattern or to find the pattern that it belongs to.
$\begingroup$
$\endgroup$
1
Regression and classification can work on some common problems where the response variable is respectively continuous and ordinal.
But the result is what would make us choose between the two.
For example, simple/hard classifiers (e.g. SVM) simply try to put the example in specific class (e.g. whether the project is "profitable" or "not profitable", and doesn't account for how much), where regression can give an exact profit value as some continuous value.
However, in the case of classification, we can consider probabilistic models (e.g. logistic regression), where each class (or label) has some probability, which can be weighted by the cost associated with each label (or class), and thus give us with a final value on basis of which we can decide to put it some label or not. For instance, label $A$ has a probability of $0.3$, but the payoff is huge (e.g. 1000). However, label $B$ has probability $0.7$, but the payoff is very low (e.g. $10$). So, for maximizing the profit, we might label the example as label $A$ instead of $B$.
Note: I am still not an expert, perhaps someone would rectify if I am wrong in some part.
-
$\begingroup$ +1 Predicting probability and cost of misclassification are enormously important! Misunderstandings of these notions cause untold amounts of frustration for data scientists. $\endgroup$– DaveCommented Jun 22, 2022 at 20:02
$\begingroup$
$\endgroup$
http://www.differencebetween.com/difference-between-classification-and-vs-regression/
Classification trees have dependent variables that are categorical and unordered.
Regression trees have dependent variables that are continuous values or ordered whole values.
$\begingroup$
$\endgroup$
1
Regression means to predict the output value using training data.
Classification means to group the output into a class.
For example, we use regression to predict the house price (a real value) from training data and we can use classification to predict the type of tumor (e.g. "benign" or "malign") using training data.
-
1$\begingroup$ It's funny when you read this answer while referring to a machine learning course at Coursera ;) . $\endgroup$– 0xc0deCommented Feb 21, 2016 at 6:58
$\begingroup$
$\endgroup$
1
Regression is defined as $E[Y \mid X]$ (i.e. the expectation of $Y$ given $X$).
A subset of these types of models where $Y$ is binary or categorical, including logistic regression and multinomial regression along with many machine learning algorithms that essentially have the same target (such as classification trees), are useful for classification.
-
$\begingroup$ Can you provide a reference for this definition? $\endgroup$– ado sarCommented Oct 20, 2023 at 15:17
$\begingroup$
$\endgroup$
It is important to be clear when using terms like regression, classification and prediction to discriminate between the task you are performing and the method used to perform it. A classification task involves taking an input and labelling it as belonging to a given class, so the output is categorical. On the other hand, a prediction task involves predicting a continuous valued output.
Methods for achieving these tasks include regression, in which a continuous valued output is estimated (or, rather, the expected value of a distribution on a continuous variable is estimated, conditional on a given set of input values). This can be used to carry out a prediction task, as you would expect. It can also be used to carry out a classification task, for example using logistic regression to estimate the log odds of the input pattern belonging to a given class. In this case, the task is classification, the method is regression.
Classification methods simply generate a class label rather than estimating a distribution parameter. K nearest neighbour is a good example where the task and the method are both called classification.
$\begingroup$
$\endgroup$
Regression: What is the temperature tomorrow?
Classification: Is it Hot or Cold tomorrow?
