Finding the derivative of the infinity norm I know that infinity/maximum norm is defined as
$$\left\| \mathbf{x} \right\| _\infty := \max_{i} \left| x_i \right|,$$
and the partial derivative of any p-norm is defined as
$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _p = \frac{x_k \left| x_k \right| ^{p-2}} { \left\| \mathbf{x} \right\| _{p}^{p-1}}.$$
So, the partial derivative of the 1-norm is
$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _1 = \frac{x_k}{ \left| x_k \right|}, $$
and the partial derivative of the 2-norm is
$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _2 = \frac{x_k}{ \left\| \mathbf{x} \right\| _2}. $$
Although, I'm not sure how to compute the partial derivative of the infinity norm through this form. Using the above form
$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _\infty = \frac{x_k \left| x_k \right| ^{\infty-2}} { \left\| \mathbf{x} \right\| _{\infty}^{\infty-1}}, $$
which would reduce to
$$\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _\infty = \frac{x_k}{ \left\| \mathbf{x} \right\| _\infty}=\frac{x_k}{\max_{i} \left| x_i \right|}.$$
Is there a separate way to analyze the partial derivative for the infinity norm?
 A: You can get the correct formula by passing to the limit $p\to \infty$ in
$$
\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _p = \frac{x_k \left| x_k \right| ^{p-2}} { \left\| \mathbf{x} \right\| _{p}^{p-1}}
$$
This doesn't mean putting $\infty$ instead of $p$; limits don't work that way. Consider the case when the maximum in the definition of $\|x\|_\infty$ is attained by one coordinate, say $x_j$. 
$$
\lim_{ p\to\infty}  \sum_{i} \frac{|x_i|^p}{\|x\|^p_\infty } = 1
$$
because one term is $1$ and the rest tend to $0$. Raising that to power $(p-1)/p$ we get  $\|x\|_p^{p-1}/\|x\|_\infty^{p-1} \to 1$. 
Now consider
$$
\frac{x_k \left| x_k \right| ^{p-2}} { \left\| \mathbf{x} \right\| _{p}^{p-1}} = 
\frac{x_k \left| x_k \right| ^{p-2} / \|x\|_\infty^{p-1} } { \left\| \mathbf{x} \right\| _{p}^{p-1} / \|x\|_\infty^{p-1}}
$$
The numerator    either tends to $0$ if $k\ne j$, or stays equal to $\operatorname{sign} x_j$ if $k=j$. Conclusion: at the points $x$ such that the norm is attained by just one coordinate $j$, we have
$$
\frac{\partial}{\partial x_k} \left\| \mathbf{x} \right\| _\infty = \operatorname{sign}(x_j) \delta_{kj}
$$
where $\delta_{kj}$ is the Kronecker delta. 
The reason this looks different from other norms is that at most places, $\infty$ norm depends on just one coordinate; the derivatives with respect to other coordinates are zeros. This isn't as easy to describe by an algebraic formula.
