matrix operator norm and inner product Is it true that $\Vert A\Vert:=\sup_{\Vert x\Vert=1}\Vert Ax\Vert=\sup_{\Vert x\Vert=\Vert y\Vert=1}\vert\langle y,Ax\rangle\vert$ for arbitrary matrices $A$?
Showing $"\geq"$ seems to be straightforward using Cauchy-Schwarz.
 A: The other direction is also short but I will provide all details. Notice that letting $y = \frac{Ax}{\|Ax\|}$ in your expression and taking the supremum over all $x$ gives
$$
\sup_{\|x\| = 1} \left|\left\langle \frac{Ax}{\|Ax\|}, Ax\right\rangle\right| \leq \sup_{\|x\| = \|y\| = 1} |\langle y, Ax\rangle|
$$
by definition of a supremum because the set on which we take the supremum on the left hand side is $\left\{(x, y) : \|x\| = 1, y = \frac{Ax}{\|Ax\|}\right\}$ which is a subset of the set on which we take the supremum on the right hand side which is $\{(x, y): \|x\| = \|y\| = 1\}$.
Now simplifying the left hand side gives
$$
\sup_{\|x\| = 1} \left|\left\langle \frac{Ax}{\|Ax\|}, Ax\right\rangle\right| = \sup_{\|x\| = 1} \frac{1}{\|Ax\|} |\langle Ax, Ax\rangle| = \sup_{\|x\| = 1} \frac{1}{\|Ax\|} \|Ax\|^2 = \sup_{\|x\| = 1} \|Ax\| = \|A\|
$$
which finishes the proof after using it in the above inequality.
Note that this works for all bounded operators on any normed vector space.
A: Suppose that $z$ is such that $\|Az\| = \|A\|$.  Then
$$
\sup_{\|x\|=1,\|y\|=1} |\langle x,Ay \rangle| \geq 
\left|\langle Az/\|A\|,Az \rangle \right| = 
\frac 1{\|A\|}
\left|\langle Az,Az \rangle \right| = \|A\|
$$
This works in the finite dimensional case where the sup is really a max.  In the infinite dimensional case, it still suffices to choose $z$ such that $\|Az\| \geq \|A\| - \epsilon$ for $\epsilon > 0$.
