I'll start by presenting my current understanding of regularized loss minimization.
In the context of ML, generally we are trying to minimize directly an empirical loss, something of the form $$\underset{h\in\mathcal{H}}{\arg\min}\sum_{i=1}^{m}L\left(y_{i},h\left(x_{i}\right)\right)$$
where there are $m$ samples $x_{1},\ldots,x_{m}$ labeled by $y_{1},\ldots,y_{m}$ and $L$ is some loss function.
In regularized loss minimization we are trying to minimize $$\underset{h\in\mathcal{H}}{\arg\min}\sum_{i=1}^{m}L\left(y_{i},h\left(x_{i}\right)\right)+\lambda\mathcal{R}\left(h\right)$$
where $\mathcal{R}:\mathcal{H}\to\mathbb{R}$ is some regularizer, which penalizes a hypothesis $h$ for it's complexity, and $\lambda$ is a parameter controlling the severity of this penalty.
In the context of linear regression, where in general we'd like to minimize $$\underset{\vec{w}\in\mathbb{R}^{d}}{\arg\min}\left\Vert \vec{y}-X^{T}\vec{w}\right\Vert _{2}^{2}$$
with ridge regularization we add the regularizer $\left\Vert \vec{w}\right\Vert _{2}^{2}$ to obtain $$\left\Vert \vec{y}-X^{T}\vec{w}\right\Vert _{2}^{2}+\lambda\left\Vert \vec{w}\right\Vert _{2}^{2}$$ And with LASSO, we use $$\left\Vert \vec{y}-X^{T}\vec{w}\right\Vert _{2}^{2}+\lambda\left\Vert \vec{w}\right\Vert _{1}$$
The LASSO solution is supposed to result in a sparse solution vector $\vec{w}$ rather then one with small entries, as is expected to happen in Ridge.
As an explanation for this last statement, I've seen the following (From "Elements of Statistical Learning"):
The ellipses are the contours of the least squares loss function and the blue shapes are the respective norm balls. The explanation (from my lecture notes)
“Since the $\ell_{1}$ ball has corners on the main axes, it is “more likely” that the Lasso solution will occur on a sparse vector”
So my questions are
Why is solving these problems equivalent to finding a point on the smallest possible contour that intersects the respective norm ball?
Why is it more likely that this will happen on a “corner” for the $\ell_{1}$ norm?