1
$\begingroup$

This is about Lipschitz constant and a function $f$ is defined and continuous on an interval $[a,b]$, and is differentiable on the interior $(a,b)$ and $x,y \in [a, b]$. I understand you can write:

$$|f(x) - f(y)| \le M |x - y|$$

where $M$ is the smallest value for which the previous inequality is true, called the Lipschitz constant. Building on this, assume we have two different functions $f_1$ and $f_2$ then $$|f_1(x) - f_1(y)| \le M_1 |x - y|$$ and $$|f_2(x) - f_2(y)| \le M_2 |x - y|$$

My questions:

  1. If $M_1 > M_2$, can we compare the 2 Lipschitz constants ($M_1$ and $M_2$) for two different functions? Is it even correct to compare them?
  2. What kind of inferences can we make if $M_1 > M_2$?
  3. Assume $f_1$ and $f_2$ are non-convex functions (machine learnt models), since $M_1 > M_2$, can we say that $f_1$ has converged to a better minima than $f_2$?
$\endgroup$
8
  • $\begingroup$ This movie-graphic may help. Larger Lipschitz constants correspond to the requirement of having to use cones with larger aperture angles. $\endgroup$ – Dave L. Renfro Sep 12 '19 at 9:52
  • $\begingroup$ @DaveL.Renfro, i see. is that all we can say? Can we infer anything more about 𝑓1 and 𝑓2 if 𝑀1 is greater than 𝑀2? I am asking this from a machine learning and function optimization perspective. $\endgroup$ – Srikar Appalaraju Sep 12 '19 at 15:26
  • 2
    $\begingroup$ I don't know anything about machine learning (except maybe this from 1991), but you can think of the Lipschitz constant as a bound on how rapidly a function's value can change when you change its input values. Lipschitz continuous means that the function's values can't increase or decrease more than some constant times the change in its input values, and the larger the Lipschitz constant, the more the function's values are allowed to change (i.e. $M_1 > M_2$ means $f_1$ can increase/decrease more rapidly than $f_2).$ $\endgroup$ – Dave L. Renfro Sep 12 '19 at 19:23
  • $\begingroup$ @DaveL.Renfro haha love the terminator reference :) Your explanation made sense to me. Pertaining to what you wrote, where can I find more theory on this? i.e. larger the Lipschitz constant, the more the function's values are allowed to change. I am assuming there are some proofs on this. $\endgroup$ – Srikar Appalaraju Sep 13 '19 at 4:07
  • $\begingroup$ I googled intuitive + "Lipschitz continuity" and found these that I believe will be helpful: Intuitive idea of the Lipschitz function AND Understanding Lipschitz Continuity AND What is the intuition behind uniform continuity?. Note: Uniform continuity on a set is when, given an "epsilon", we can find a single "delta" that works for that "epsilon" (continued) $\endgroup$ – Dave L. Renfro Sep 13 '19 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.