Equivalent Definitions of the Operator Norm How do you prove that these four definitions of the operator norm are equivalent?
$$\begin{align*}
\lVert A\rVert_{\mathrm{op}} &= \inf\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}\\
&=\sup\{ \lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert\leq 1\}\\
&=\sup\{\lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert = 1 \}\\
&=\sup\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}.
\end{align*}$$
 A: Let $$\begin{align*}
I &= \inf\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}\\
S_1&=\sup\{ \lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert\leq 1\}\\
S_2&=\sup\{\lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert = 1 \}\\
S_3&=\sup\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}.
\end{align*}$$
Notice that $S_2 \le S_1$ and as $\|Av\| /\|v\| = \| A(v / \|v\|)\|$ we have $S_3 \le S_2$. Now if $\|v\|\le 1$ we have $\|Av\| \le \|Av\| /\|v\|$. Then $S_1 \le S_3$ and
$$ S_1=S_2=S_3.$$
Now note that 
$$ \|Av\| \le S_3 \|v\| \quad \forall v \in V.$$
 Then $I \le S_3$ and by definition of $\sup$ we have
$$ I \ge \|Av_n\| /\|v_n\| \ge S_3 - 1/n \quad \forall n.$$
Then $S_3 = I$.
A: The four definitions are equivalent only if $V\neq\{0\}$ (which, admittedly, is usually the case).  
Reason: When $V=\{0\}$, there is no $v\in V$ such that $\|v\|=1$.
So $\sup\varnothing = -\infty$, but we clearly want $\|A\|=0$ if $A$ is the zero operator. This (small but still) mistake can be found in many textbooks.
In summary, only the first two expressions make sense. 
A: Another equivalent definition, which I can rarely see:
$$\lVert A\rVert=\sup_{\lVert x \rVert < 1} \lVert Ax \rVert$$
It's easy to see that
$$\sup_{\lVert x \rVert < 1} \lVert Ax \rVert \leqslant \sup_{\lVert x \rVert \leqslant  1} \lVert Ax \rVert$$
For the other direction, let $(r_n)$ be a sequence of real numbers, with $0 \leqslant r_n < 1$ and $r_n \to 1$, and $x \in V$ with $\lVert x \rVert \leqslant 1$. Then, we have that $\lVert r_n x\rVert<1$, which implies that $\lVert A(r_n x) \rVert \leqslant \sup_{\lVert x \rVert < 1} \lVert Ax \rVert$, but $\lVert A(r_n x) \rVert \to \lVert A x\rVert$, which implies that $\lVert Ax \rVert \leqslant \sup_{\lVert x \rVert < 1} \lVert Ax \rVert$.
A: Let $a=\inf\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\},b=\sup\{ \lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert\leq 1\},c=\sup\{\lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert = 1 \},d=\sup\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}.$
$a=\inf\{ c\;\colon\; \lVert Av\rVert\leq c\lVert v\rVert \text{ for all }v\in V\}= \inf\{ c\;\colon\;\frac{\lVert Av\rVert}{\lVert v\rVert}\leq c \text{ for all }v\neq 0\}$ (easily verified), which is the infimum of the set of upper bounds for the set $\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}$, i.e., the least upper bound. So $a=d$.
Next see that $\frac{\lVert Av\rVert}{\lVert v\rVert}=\lVert A(\frac{v}{\lVert v\rVert})\rVert$, and $\frac{v}{\lVert v\rVert}$ has norm $1$. And if $v$ has norm $1$, then we can write $v =\frac{v}{\lVert v\rVert}.$ This shows that $c=d$. Finally I'll show that $b=c$ (because I find this more interesting). $b\geq c$ is easily seen. For each $v$ with $\lVert v\rVert\leq 1$, there is $\tilde v=\frac{v}{\lVert v\rVert}$ having norm $1$, with $\lVert A\tilde v\rVert=\frac{\lVert Av\rVert}{\lVert v\rVert}\geq \lVert Av\rVert.$ This proves $c\geq b.$
A: Remember that if $s=\sup X$ and $x\leq t$ for all $x\in X$, then $s\leq t$. Also, if $s=\inf X$ and $t\in S$, then $s\leq t$. Now, let us write
$$\begin{align*}
a&= \inf\{ k\;\colon\; \lVert Av\rVert\leq k\lVert v\rVert \text{ for all }v\in V\},\\
\\
b&=\sup\{ \lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert\leq 1\},\\
\\
c&=\sup\{\lVert Av\rVert\;\colon\; v\in V\text{ with }\lVert v\rVert = 1 \},\\\\
d&=\sup\left\{ \frac{\lVert Av\rVert}{\lVert v\rVert}\;\colon\; v\in V\text{ with }v\neq 0\right\}.
\end{align*}$$
Note that:


*

*$\|Av\|\leq a\|v\|$ for all $v\in V$. Taking $v\in V$ with $\|v\|\leq 1$ we obtain $\|Av\|\leq a$ and thus $b\leq a$;

*$\|Av\|\leq b$ for all $v\in V$ with $\|v\|\leq 1$. Take $v\in V$ with $\|v\|=1$ and define $v_n=(1-1/n)v$. Since $\|v_n\|=1-1/n\leq 1$, we conclude that $\|Av_n\|\leq b$ for all $n\in\mathbb{N}$. Taking the limit we obtain $\|Av\|\leq b$ and thus $c\leq b$;

*$\|Av\|\leq c$ for all $v\in V$ with $\|v\|= 1$. Taking $v\in V$ with $v\neq 0$, we obtain $\left\|\frac{v}{\|v\|}\right\|=1$. Hence $\frac{\|Av\|}{\|v\|}=\left\|A\left(\frac{v}{\|v\|}\right)\right\|\leq c$ and thus $d\leq c$;

*$\frac{\|Av\|}{\|v\|}\leq d$ for all $v\in V$ with $v\neq 0$. Hence $\|Av\|\leq d\|v\|$ for all $v\in V$ and thus $a\leq d$.
This shows that $a\leq d\leq c\leq b\leq a$ and thus $a=b=c=d$.
A: I'll give you part of one to give you an idea of the flavor, but you should really do them yourself.
Let $w\neq 0$. Then $\frac{1}{\lVert w\rVert}$ makes sense. Now notice that
$$A\left(\frac{1}{\lVert w\rVert}w\right) = \frac{1}{\lVert w\rVert}A(w).$$
Therefore,
$$\left\lVert A\left(\frac{w}{\lVert w\rVert}\right)\right\rVert = \frac{\lVert Aw\rVert}{\lVert w\rVert}.$$
But $\frac{w}{\lVert w\rVert}$ is a vector of norm $1$, so...
A: It is of course possible to prove direct equivalence of any two of the definitions as well. Here is one example.
Let
$$ 
\begin{align*}
a &= \inf \left\{ 
k > 0: 
\left\lVert L v \right\rVert \leq k \left\lVert v \right\rVert
\text{ for all } 
v \in V 
\right\} \text{ and} 
\\
b &= \sup \left\{ 
\left\lVert L v \right\rVert 
\text{ for all } 
v \in V 
\text{ with }
\left\lVert v \right\rVert \leq 1
\right\} .
\end{align*}
$$
Now we prove that $a = b$ by proving that $b \leq a$ and $a \leq b$.
First, we have that for any $v \in V$
$$
\left\lVert L v \right\rVert \leq a \left\lVert v \right\rVert ,
$$
since $a$ was the infimum of all numbers that satisfies this equation.
In particular, for any $v$ with $ \left\lVert v \right\rVert \leq 1$ we have
$$
\left\lVert L v \right\rVert \leq a \left\lVert v \right\rVert \leq a . 
$$
This tells us that $a$ is an upper bound for $\left\lVert L v \right\rVert$ when $\left\lVert v \right\rVert \leq 1$, and hence an upper bound for the set in the definition of $b$. Since $b$ is the supremum (smallest upper bound), we have that
$$ 
b \leq a . 
$$
The opposite direction requires a simple trick. Let $v \in V$ be any vector and choose an $\epsilon < 1$. Then the vector defined by
$$
u = \frac{v}{\left\lVert v \right\rVert + \epsilon} ,
$$
fulfils
$$
\left\lVert u \right\rVert = 
\frac{\left\lVert v \right\rVert}{\left\lVert v \right\rVert + \epsilon} 
< 1 .
$$
Hence, we see that it is in the set that defines $b$, and we must have
$$
\left\lVert L u \right\rVert \leq b. 
$$
Inserting the relation between $u$ and $v$ we get
$$
\left\lVert L v \right\rVert \leq b \left(\left\lVert v \right\rVert + \epsilon \right) ,
$$
which in the limit $\epsilon \to 0$ becomes
$$
\left\lVert L v \right\rVert \leq b \left\lVert v \right\rVert .
$$
Since $a$, being the infimum of all numbers satisfying this relation, we must have
$$ 
a \leq b ,
$$
and we are done
