Definition: The Operator norm, denoted $\| \cdot \|_{\text{op}}$ is defined to be $$\| B \|_{\text{op}} = \inf\left\{ c \geq 0 : \|Bv \|\leq c\|v \|\ , \forall v \in V\right\}$$

Definition: The Frobenius norm, denoted $\|\cdot \|_F$ is defined to be $$\| A \|_F = \sqrt{\sum_{i=1}^n \sum_{j=1}^m |a_{ij}|^2}$$

Now, the inequality

\begin{align*} \|A^TB \|_F &\leq \|A \|_F \|B \|_F \end{align*}

is rather straight forward. However, obtaining $\|B \|_F \leq \|B \|_{\text{op}}$ is not. Any help would be appreciated.

  • $\begingroup$ Wait, $A$ and $B$ are both $n \times m$ matrices? Did you mean $AB^\top$ or $A^\top B$? Or are these matrices square? $\endgroup$ – Theo Bendit Sep 19 '18 at 14:07
  • $\begingroup$ Yes, sorry $A^TB$. I will edit. Thank you. $\endgroup$ – Dragonite Sep 19 '18 at 14:18
  • $\begingroup$ Why do you write $\|\ A\|$ instead of $\|A\|$? (I actually changed that in every instance.) $\endgroup$ – Michael Hardy Sep 19 '18 at 17:47

Let $A = (a_1, \dots, a_m)$ where $a_j$ denotes the columns. \begin{align*} \|A^T B\|_F^2 = \|B^T A\|_F^2 = \|( B^Ta_1, \dots, B^Ta_m)\|_F^2 = \sum_{j=1}^m \|B^T a_j\|_2^2 \\\le \sum_{j=1}^m \|B^T\|_{op}^2 \|a_j\|_2^2 = \|B^T\|_{op}^2 \|A\|_F^2. \end{align*} Note $\|B^T\|_{op} = \|B\|_{op}$.


Though not an answer to the question, I would like to mention that the inequality $\|B\|_F \leq \|B\|_{\mathrm{op}}$ is not correct, it should be $\|B\|_{\mathrm{op}}\leq\|B\|_{F}$. A quick way to see this, is to use the equations $$ \|B\|^2_{\mathrm{op}} = \lambda_{\max}(B^TB)= \sigma^2_{\max}(B) $$ and $$ \|B\|_F^2 = \operatorname{trace}(B^T B) = \sum_{i=1}^{\min\{m,n\}} \sigma^2_i (B),$$ where $\lambda_i$ and $\sigma_i$ stand for an eigenvalue and singular value respectively.

Reference: "Matrix norm" on Wikipedia.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.