1
$\begingroup$

I have a training set which is set of essays written by students for a question. These essays are all scored by human evaluators with labels such as 1, 2 , 3 which is actually marks allotted for those essays. I want know whether to use regression or classification algorithm for machine learning purposes! My readings on Machine learning suggests me to go with classification algorithm but again should I go with numeric classification or nominal classification. I am thinking numeric - Am I correct?

$\endgroup$
  • 1
    $\begingroup$ Occam's Razor. Try something simple to code first like a regression/perceptron. If that doesn't work, try something more complicated like a logistic regression. If that doesn't work try an support vector machine. $\endgroup$ – Alex R. Oct 21 '13 at 15:21
  • $\begingroup$ Thanks Alex. So the only way to decide on the best fit algorithm is trial and error? I am surprised. I thought there will be clear cut definitions on when to use a particular algorithm. $\endgroup$ – SUNIL KUMAR C Oct 21 '13 at 15:26
  • 1
    $\begingroup$ It all depends on what you are trying to measure and the kind of error you are willing to allow for. Some data is clustered nicely allowing for a multi-class perception or a linear regression. Other times, it's really mixed together so you need to unravel it something akin to a support vector machine. For something like this, perhaps a mixture of gaussians would be appropriate if you have strong but spread out clustering for each label. $\endgroup$ – Alex R. Oct 21 '13 at 15:36
0
$\begingroup$

Since your labels are discrete, you are looking at a classification problem with text data. I recommend using Logistic Regression (LR) as implemented in scikit-learn library in Python.

You would need to construct $A_{V\times D}$ term-document matrix where $V$ is the vocabulary size and $D$ is the number of documents. Thus, each column of $A$ is a document or a collection of word-counts (here we are assuming that words are exchangeable and only their counts matter). I recommend using standard text-preprocessing such as tokenization, stop-word removal and tf-idf smoothing. Gensim library implements many of these functionalities.

The input to your Lostigic Regression (LR) becomes an $(X,y)$ pair where $X=A^{T}$ (each row of $X$ is a document with $V$ features) and $y \in \{1,...,K\}$ is the label. Once trained, you can use LR to predict the label of a new document (pre-processed in the same way as the training data).

$\endgroup$
  • $\begingroup$ Welcome to mathSE. Seems like a good answer. $\endgroup$ – 6005 Jun 30 '17 at 3:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.