How to minimize $\|x\|_1$ when $\|x\|_2 = 1, x_i \in [0,1]$ I'm looking for a solution that minimizes 
$$
f(x) = \|x\|_1
$$
where $\|x\|_2 = 1, x_i \in [0,1]$ ($i=1,\dots,K$). The thing is if I try Lagrange multiplier method, I get $1/\sqrt{K}$. But intuitively the solution must be one of standard basis elements $(0,.\dots,1,\dots,0)$. 
How can I possibly derive that? Can I not use Lagrange multiplier because $1$-norm is not differentiable at zero?
 A: You can show that the 2-norm is a lower bound on the 1-norm. To see this, note that $|x_1|^2 + |x_2|^2 + \ldots + |x_K|^2 \leq (|x_1| + |x_2|+ \ldots + |x_K|)^2$ and the left hand side is $\lVert x\rVert_2^2$ and the right hand side is $\lVert x \rVert_1^2$. So, $\lVert x \rVert_2 \leq \lVert x \rVert_1$.
Now, you know your constraint is $\lVert x \rVert_2=1$, so $f(x) \geq 1$. The standard basis achieves the above inequality with equality, so the minimum value of $f(x)$ is $1$.

As an aside, you can also prove the inequality $\lVert x\rVert_1 \leq \sqrt{K} \lvert x\rVert_2$. Let $v$ be the vector made by taking the absolute value of $x$ componentwise and $\mathbf{1}$ the allones vector. Then, $\lvert x \rvert_1 
 = \mathbf{1} \cdot v  \leq \sqrt{K} \lvert x \rvert_2$ by the Cauchy Schwarz inequality. By the constraint  $\lVert x \rVert_2=1$, you see $\lvert x \rvert_1 \leq \sqrt{K}$. Now, the vectors you've found, where $x_i = \frac{1}{\sqrt{K}}$ meet this upper bound on $\lvert x \rvert_1$ with equality. 
This goes to the comment left above; stationary points of the Lagrangian are necessary (but not sufficient) for extrema for differentiable problems. 

A farther aside: On a finite dimensional vector space, all norms are equivalent; that is, you can upper and lower bound norm A by  constant multiples of norm B, for arbitrary norm A, B. This is a standard exercise for undergraduate math majors. 
