Largest singular value of non square matrix Let $B$ be an $m\times n$ matrix with complex number as its element.
Let $\sigma$ denotes the largest singular value of $B$
Prove that 
\begin{equation}
\sigma = \max\limits_{\|u\|_2=1,\|v\|_2=1} |u^*Bv|.
\end{equation}
My partial solution:
Let $x$ be the eigenvector of $B^*B$ that corespond to eigen value $\sigma^2$.
Define $z=\frac{x}{\|x\|_2}$ and $w=\frac{1}{\sigma} Bz$.
Note that $\|z\|_2=1$ and $\|w\|_2=\sqrt{w^*w}=\sqrt{\frac{1}{\sigma^2} z^*B^*Bz}=\frac{1}{\sigma}\sqrt{z^*\sigma^2 z}=1$.
Thus
\begin{align*}
\sigma=\frac{1}{\sigma} z^*\sigma^2z=\frac{1}{\sigma} z^* B^*Bz=w^*Bz\leq \max\limits_{\|u\|_2=1,\|v\|_2=1} |u^*Bv|.
\end{align*}
My question is that how to prove the reverse inequality.
If $m=n$, then I can solve the problem by using the Rayleigh-Ritz theorem for $B^*B$. But, I have no idea for $m\neq n$.
Thank you.
 A: What about this?
$$u^* B v \leq \|u\| \times \|B v\|= \sqrt{v' B* B v} \leq \sigma,$$
where the last weak inequality follows from the fact that $B^*B$ is symmetric and hence can be written as $\sum_i \lambda_i x_i x_i'$ where $\lambda_i$ are its eigenvalues and $x_i$ its eigenvectors.
A: In what follows, we use, for the inner product of vectors, the notation $\langle u,v\rangle=v^*u$.
By definition
$$
\sigma^2=\max_{\|v\|_2=1} \langle v, B^*B v\rangle=\max_{\|v\|_2=1} \langle Bv, B v\rangle=\max_{\|v\|_2=1}\|Bv\|^2,
$$
and hence
$$
\sigma=\max_{\|v\|_2=1}\|Bv\|. \tag{1}
$$
At the same time, for any vector $w\in\mathbb R^m$ we know that
$$
\|w\|=\max_{\|u\|_2=1}\langle w,u\rangle. \tag {2}
$$
Combining $(1)$ and $(2)$ we obtain
$$
\sigma=\max_{\|v\|_2=1}\|Bv\|=\max_{\|v\|_2=1,\|u\|_2=1}\langle Bv,u\rangle.
$$
A: Let $B = U S V^*$ be the singular value decomposition of $B$ with $S$ is the diagonal matrix of the singular values of $B$, then for all unit vectors $u$ and $v$, let $u' = U^*u$ and $v' = V^*v$:
$$
\begin{align*}
 u^* B v& = u^* U S V^* v = u'^* S v'
= \sum_i u'_i S_{ii} v'_i \\
\therefore \;
\left|u^* B v\right|
&= \left|\sum_i u'_i S_{ii} v'_i\right|
\leq \left|\sum_i u'_i \sigma v'_i\right|
= \sigma\,\left|u' \cdot v'\right|
\leq \sigma
\end{align*}
$$
since $\forall i \; S_{ii} \leq \sigma$ by assumption and $\left|u' \cdot v'\right| \leq 1$ since $\|u'\|_2 = \|U^*u\|_2 = 1$ and $\|v'\|_2 = \|V^*v\|_2 = 1$  because $U$ and $V$ are unitary by definition of the SVD.
To also prove that the maximum actually is $\sigma$ assume $u$ and $v$ are such that $\forall i \; u'_i = v'_i = \delta_{ia}$ for a fixed $a$ where $S_{aa} = \sigma$ then
$$
u^*Bv
= \sum_i u'_i S_{ii} v'_i
= u'_a S_{aa} v'_a
= \sigma
$$
Since unitary matrices are invertible, there always exist unit vectors $u$ and $v$ such that
$\left| u^* B v \right| = \sigma$
A: The reverse inequality follows from the fact that $\|B\|_2 = \sigma$.
Let $u$ and $v$ be unit vectors. Then
\begin{align*}
| u^* B v | &\leq \|u \|_2 \|B v\|_2 \qquad \text{(by Cauchy-Schwarz inequality)}\\
&\leq \|u\|_2 \|B\|_2 \|v\|_2 \\
&= \sigma.
\end{align*}
It follows that 
\begin{equation}
\max_{\|u\|_2=1,\|v\|_2=1} | u^* B v| \leq \sigma.
\end{equation}
