Scaling of objective function in optimization problems. In a lot of optimization algorithms I have seen that they use a scaling factor to scale the objective function. I dont understand what is the reason behind this. Why do we need scaling of objective function in optimization algorithms? Does it work without scaling. Logically it should work but I am a litte bit confused now. 
I hope you can answer me and thank you indeed
 A: When you study a optimization problem theoretically, scaling does not matter. However, sometimes one adds a scaling factor, such that the derivative is a little bit simpler, e.g., instead of $x^2$ one likes to minimize $\frac12 x^2$.
However, scaling is of importance for the numerically solution. If your optimization method is not scaling invariant, then you get a different sequence of iterates (not only due to rounding errors). Newton's method, e.g., is affine invariant, i.e., if you do an affine transformation of your optimization problem, you get (up to the transformation) the same sequence of iterates.
A: Beyond the numerical efficiency perspective there is also an impact on the model.
If your feature variables are scaled then your objective function needs to be scaled, otherwise you would need disproportionately large coefficients to get to the same target value.
Scaling on power law distributed objective functions also give skewed results.
These objective functions require sublinear transformations.
From Skiena's Data Science Design Manual p276:
"Small-scale variables need small-scale targets, in order to be realized using small-scale
coefficients. Trying to predict GDP from Z-scored variables will require
enormously large coefficients. How else could you get to trillions of dollars from
a linear combination of variables ranging from -3 to +3?"
