Is it true: $||A||_2 = \min\{ ||A||_1 ,||A||_3,||A||_4,\ldots \ldots, ||A||_{\infty},\|A\|_F\} $? While running one algorithm , I observed the following peculiar relationship (at-least to me). I am not quite sure whether it is true in general, but I could not succeeded either in producing any counter example to nullify it.
Let $A$ be a square matrix;
$$\|A\|_2 \leq \|A\|_1\quad,$$
$$||A||_2 \leq ||A||_{\infty}\quad,$$
$$||A||_2 \leq ||A||_F\quad.$$
Where $||A||_1,||A||_2,||A||_{\infty}$ and $||A||_F$ are 1-norm, 2-norm, infinity norm and frobenius norm respectively.
Are above relationships indeed true? and if so, then is it correct to say that (2-norm) always lower bounds all the other norms. i-e $||A||_2 \leq ||A||_1 ,||A||_3,||A||_4,\ldots \ldots, ||A||_{\infty},||A||_F $ ?
Would be thankful for any suggestion. 
 A: All norms on a finite dimensional vector space are equivalent (that is, for any two norms A and B, there exist universal constants $C_1, C_2,C_3, C_4$ such that $C_1 || \cdot ||_A \leq || \cdot ||_B \leq C_2 || \cdot ||_A$ and  $C_3 || \cdot ||_B \leq || \cdot ||_A \leq C_4 || \cdot ||_B$). 
But this by no means tells you one norm is always the smallest in a family of norms (without knowing the constants, anyway). A cute result from these notes related to Fritz John's ellipsoid theorem tells you on a n-dim real normed vector space with norm A, you can always find an inner product norm $B$ with $C_1 = 1$ and $C_2 = \sqrt{n}$ where these constants are indeed tight.
I just used Octave to generate some random matrices using the code:
octave:6> A=randn(2); norm(A)-norm(A,1)
ans =  0.023753
octave:7> A
A =
1.4542  -1.0416
-1.7263   2.0556
to get a counter example.
As for the other way around, 
octave:15> A=randn(2); norm(A)-norm(A,1)
ans = -0.70221
octave:16> A
A =
0.30553  -1.79793
0.41323   0.93044
