8
$\begingroup$

After reading through a few references, I have come to know that for machine learning in general, it is necessary to normalize features so that no features are arbitrarily large ($centering$) and all features are on the same scale ($scaling$).

However, I'm having a bit of difficulty in visualizing the impact of this for a decision tree. Does data normalization impact the decision tree structure? If yes, how?

$\endgroup$
1
  • $\begingroup$ If a column you are using is a combination of two others, then it might be a good idea to normalise those beforehand (or rescale in some manner so that each column contributes in the way you want to the final value). $\endgroup$
    – Guillermo Mosse
    Sep 16, 2019 at 8:23

1 Answer 1

Reset to default
11
$\begingroup$

Normalization should have no impact on the performance of a decision tree. It is generally useful, when you are solving a system of equations, least squares, etc, where you can have serious issues due to rounding errors. In decision tree, you are just comparing stuff and branching down the tree, so normalization would not help.

$\endgroup$
2
  • $\begingroup$ Is there a way I can prove this formally? Any reference to this would help. I'm thinking data would be centered and scaled but structure would remain the same. However, I need to prove this. Intuitively, I'm thinking I should show there's no information gain change. $\endgroup$
    – Raaj
    Sep 11, 2014 at 17:28
  • $\begingroup$ Go to section on advantages of decision tree : en.wikipedia.org/wiki/Decision_tree_learning $\endgroup$
    – vdesai
    Sep 11, 2014 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.