Examples on where to prefer $L^1$ norm over the squared $L^2$ norm In Deep Learning (page 39) it is stated that:

The squared $L^2$ norm is more convenient to work with mathematically and computationally than the $L^2$ norm itself. For example, the derivatives of the squared $L^2$ norm with respect to each element of $\mathbf x$ each depend only on the corresponding element of $\mathbf x$, while all of the derivatives of the $L^2$ norm depend on the entire vector. In many contexts, the squared $L^2$ norm may be undesirable because it increases very slowly near the origin.
In several machine learning applications, it is important to discriminate between elements that are exactly zero and elements that are small but nonzero. In these cases, we turn to a function that grows at the same rate in all locations, but retains mathematical simplicity: the $L^1$ norm.

Can you list some examples in which it happens that the $L^2$ norm is undesirable (possibly in the context of Deep Learning)?
 A: He is referring to the fact that the derivatives $L^2$ norm depend on the whole vector. For example, let 
$$f(x) = \|x\|_2 = \sqrt{x_1^2 + \dots + x_n^2}$$
$$g(x) = \|x\|_2^2 = x_1^2 + \dots + x_n^2$$
Then 
$$\frac{\partial f}{\partial x_i} = \frac {x_i}{\|x\|_2}$$
$$\frac{\partial g}{\partial x_i} = 2x_i$$
Notice how the to compute the partial derivative of $g$, you only need the $i$-th component. For $f$, you need the whole vector because you need to compute $\|x\|_2$. If you have a vector $x$ with 10.000.000 elements, this could result in a big difference (depending on what you need the partial derivatives for an how you handle them). 
For the relation with the $L^1$ norm; it is simply saying that the $L^1$ norm likes sparse vectors (hence sets some components exactly to $0$). Now "sets" doesn't really mean anything since it's not like the $L^1$ norm takes decisions, but in several optimisation problems if you use the $L^1$ norm you end up with sparse vectors. A prime example is the lasso regression, or in general an optimizations problem where a regularizer in the form $\lambda \|x\|_1$ is added. The solutions to the latter will be usually sparser than when using a regularizer in the form $\lambda \|x\|_2$
