# Finding the derivative of the infinity norm

I know that infinity/maximum norm is defined as

$$\left\| \mathbf{x} \right\| _\infty := \max_{i} \left| x_i \right|,$$

and the partial derivative of any p-norm is defined as

$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _p = \frac{x_k \left| x_k \right| ^{p-2}} { \left\| \mathbf{x} \right\| _{p}^{p-1}}.$$

So, the partial derivative of the 1-norm is

$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _1 = \frac{x_k}{ \left| x_k \right|},$$

and the partial derivative of the 2-norm is

$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _2 = \frac{x_k}{ \left\| \mathbf{x} \right\| _2}.$$

Although, I'm not sure how to compute the partial derivative of the infinity norm through this form. Using the above form

$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _\infty = \frac{x_k \left| x_k \right| ^{\infty-2}} { \left\| \mathbf{x} \right\| _{\infty}^{\infty-1}},$$

which would reduce to

$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _\infty = \frac{x_k}{ \left\| \mathbf{x} \right\| _\infty}=\frac{x_k}{\max_{i} \left| x_i \right|}.$$

Is there a separate way to analyze the partial derivative for the infinity norm?

• Try using the very definition of your norm. Mar 17, 2018 at 21:22
• Unfortunately, I still come up with the same answer. I cannot see a different answer other that the specific $x_k$ divided by the maximum of $\left| x_k \right|$. Mar 17, 2018 at 22:24
• math.stackexchange.com/questions/437039/… Mar 18, 2018 at 9:30

You can get the correct formula by passing to the limit $p\to \infty$ in $$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _p = \frac{x_k \left| x_k \right| ^{p-2}} { \left\| \mathbf{x} \right\| _{p}^{p-1}}$$ This doesn't mean putting $\infty$ instead of $p$; limits don't work that way. Consider the case when the maximum in the definition of $\|x\|_\infty$ is attained by one coordinate, say $x_j$. $$\lim_{ p\to\infty} \sum_{i} \frac{|x_i|^p}{\|x\|^p_\infty } = 1$$ because one term is $1$ and the rest tend to $0$. Raising that to power $(p-1)/p$ we get $\|x\|_p^{p-1}/\|x\|_\infty^{p-1} \to 1$. Now consider $$\frac{x_k \left| x_k \right| ^{p-2}} { \left\| \mathbf{x} \right\| _{p}^{p-1}} = \frac{x_k \left| x_k \right| ^{p-2} / \|x\|_\infty^{p-1} } { \left\| \mathbf{x} \right\| _{p}^{p-1} / \|x\|_\infty^{p-1}}$$ The numerator either tends to $0$ if $k\ne j$, or stays equal to $\operatorname{sign} x_j$ if $k=j$. Conclusion: at the points $x$ such that the norm is attained by just one coordinate $j$, we have $$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _\infty = \operatorname{sign}(x_j) \delta_{kj}$$ where $\delta_{kj}$ is the Kronecker delta.
The reason this looks different from other norms is that at most places, $\infty$ norm depends on just one coordinate; the derivatives with respect to other coordinates are zeros. This isn't as easy to describe by an algebraic formula.