I am master of artificial intelligence and trying to understand the concept of q-norm in deeps.

Here is my question, I am as a machine learning engineer using q-norm as the distance between my prediction and actual happening value. In another word, this value is my measurement that shows have much is my prediction algorithm will predict close to the actual happening. I know that from a mathematician point of view in the formula for q-norm if we consider $q=1$ we will have Manhattan distance which will be the following distance between two points as a geometric representation.

Manhattan Distance geometric representation

The formula for this distance is the following: $$ \text{Manhattan Distance}(X_{real},X_{prediction})=||X_{real}-X_{prediction}||_1= |x_{real, 1} - x_{prediction,1}|+|x_{real, 2} - x_{prediction,2}|+ \dots + |x_{real, n} - x_{prediction,n}| $$

Also, if we consider $q=2$ the distance between our prediction and actual real happening value is called Euclidean distance. The following figure shows the geometric representation of this distance.

euclidean distance geometric representation

And as everybody knows this is the formula for Euclidean distance.

$$ \text{Euclidean Distance}(X_{real},X_{prediction})=||X_{real}-X_{prediction}||_{2}=\sqrt{ |x_{real, 1} - x_{prediction,1}|^2+|x_{real, 2} - x_{prediction,2}|^2+ \dots + |x_{real, n} - x_{prediction,n}|^2} $$

Now my question is what is the geometric representation if I use $q=3$? I mean Manhattan distance is like counting the number of moves to the right and number of moves to the top and Euclidean distance is the length of the direct line between two point on the plot what is the third norm distance.

  • $\begingroup$ See for example this question. $\endgroup$ – Delta-u Apr 19 '18 at 20:38
  • $\begingroup$ Possible duplicate of How to visualize norm 3, norm 4,.. distance between two points in X-Y Plane $\endgroup$ – Xander Henderson Apr 19 '18 at 20:40
  • $\begingroup$ Thanks, the answers to this similar question is not convincing because I want to know what is the geometric representation or plot of 3-norm which if I am not mistaken most be $||x-y||$ and it is a vector, not a line. the answer to the mentioned question is the plot for the line $||x|| + ||y| = 1$ $\endgroup$ – Ehsan Apr 19 '18 at 20:51

If you wanted to draw the 3-norm to your second picture for example it would be a line shorter than the straight line between the two points. And there is no neat geometric way to simply draw how much shorter it is. And it will only get smaller as q increases. That is because q-norm are non increasing with respect to q. Ultimately for $q \to \infty$ you will end up with $||.||_\infty$ which would be the max of the two segment $|p_x-q_x|=8$ here.

An interesting "machine learning"-wise interpretation is: As q increases the norm will give more importance to the coordinates (features) with the biggest differences. As q decreases the outlier become less and less important and the impact of each features is more balanced. This will greatly impact your optimum when you minimize the norm. For example, linear least square regression solution penalize a lot outliers and so would do any q-norm with $q\geq2$ compare to a 1-norm.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.