18
$\begingroup$

The Frobenius norm of a $m \times n$ matrix $F$ is defined as

$$\| F \|_F^2 := \sum_{i=1}^m\sum_{j=1}^n |f_{i,j}|^2$$

If I have $FG$, where $G$ is a $n \times p$ matrix, can we say the following?

$$\| F G \|_F^2 = \|F\|_F^2 \|G\|_F^2$$

Also, what does Frobenius norm mean? Is it analogous to the magnitude of a vector, but for matrix?

$\endgroup$
  • 2
    $\begingroup$ in general, the Frobenius norm need not be multiplicative. it is just sub multiplicative. Think about Cauchy Schwarz inequality. $\endgroup$ – user251257 Aug 11 '15 at 15:49
  • $\begingroup$ concerning the "meaning" of the norm, the set of matrices is a vector space which means up to a choice of notation it's just $\mathbb{R}^p$ for $p=n^2$. This Frobenius norm is just the natural length of the vector which is formed by stringing out the entries of the matrix into an $n^2$-vector. Furthermore, the norm makes the set of matrices a normed linear space which means you get all the excellent theorems which go with that structure. In particular, a nice theory of power series etc. $\endgroup$ – James S. Cook Aug 12 '15 at 1:57
  • $\begingroup$ If we had equality in all cases then we'd have a normed algebra, but, that is too greedy except in a few special cases like the matrices which correspond to the reals, complex or quaternions. In other cases, the existence of zero-divisors in the algebra necessarily either spoils multiplicativity of the norm or it gives a multiplicative "norm" which isn't really a norm. $\endgroup$ – James S. Cook Aug 12 '15 at 2:01
19
$\begingroup$

Actually there is $$||FG||^2_F \leqslant||F||^2_F||G||^2_F$$ The proof is like: \begin{align} \|FG\|^2_F&=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left|\sum\limits_{k=1}^nf_{ik}g_{kj}\right|^2 \\ &\leqslant\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k=1}^n|f_{ik}|^2\sum\limits_{k=1}^n|g_{kj}|^2\right)\tag{Cauchy-Schwarz} \\ &=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k,l=1}^n|f_{ik}|^2|g_{lj}|^2\right) \\ &=\sum\limits_{i=1}^{m}\sum\limits_{k=1}^{n}|f_{ik}|^2\sum\limits_{l=1}^{n}\sum\limits_{j=1}^{p}|g_{lj}|^2 \\ &=\|F\|^2_F\|G\|^2_F \end{align} Frobenius norm like vector norm and similar to $l_2$.

$\endgroup$
  • $\begingroup$ I wonder, in general, for which matrices is equality attained. $\endgroup$ – James S. Cook Aug 12 '15 at 0:55
  • 1
    $\begingroup$ When Cauchy-Schwarz becomes equal. $\endgroup$ – Math Wizard Aug 12 '15 at 1:01
  • $\begingroup$ Yes, how can we otherwise characterize such matrices... (your answer is appropriate to the question asked I think as the OP wants a general identity). $\endgroup$ – James S. Cook Aug 12 '15 at 1:25
  • $\begingroup$ I could ask a stand-alone question so this one does not get further cluttered. $\endgroup$ – James S. Cook Aug 12 '15 at 1:50
  • $\begingroup$ I mean I could ask a separate question about equality. Of course, you could just add a second answer here. $\endgroup$ – James S. Cook Aug 12 '15 at 2:10
6
$\begingroup$

Let $A$ be $m \times r$ and $B$ be $r \times n$. A better bound here is $$ \| A B \|_F \le \|A\| \|B\|_F \quad (*) $$ where $\|A\|$ is the $\ell_2$ operator norm: $$ \|A\| = \max_{\|x\|_2\, \le\, 1} \|A x\|_2. $$ It is also equal to the largest singular value of A. From this definition $\|A x\|_2 \le \|A\| \|x\|_2$ for any vector $x$ in $\mathbb R^r.$

Since $\|A\| \le \|A\|_F$, inequality (*) is a strictly better inequality than the sub-multiplicative inequality for the Frobenius norm.

To see the inequality, let $B = [b_1 \mid b_2 \mid \cdots \mid b_n]$ be the column decomposition of $B$. Then, $A B = [Ab_1 \mid A b_2 \mid \dots \mid Ab_n]$ is the column decomposition of $AB$. It follows that \begin{align*} \| A B \|_F^2 \le \sum_{j=1}^n \|A b_j\|^2 \le \|A\|^2 \sum_j \|b_j\|^2 = \|A\|^2 \|B\|_F^2. \end{align*}

$\endgroup$
0
$\begingroup$

We will prove that $$ \Vert FG \Vert_f^2 \leq \Vert F \Vert_f^2 \cdot \Vert G \Vert_f^2.$$ We have $$\Vert FG \Vert_f^2 = \mathsf{Tr}(FG G^TF^T) = \mathsf{Tr} (F^TFGG^T),$$ by the cyclic property of trace function. To prove the theorem, it is enough if we show that $$\mathsf{Tr}(F^TFGG^T) \leq \mathsf{Tr}(F^TF) \mathsf{Tr}(GG^T).$$ Observe that $F^TF$ and $GG^T$ are positive semidefinite and symmetric matrices. Hence, they are diagonalizable and can be written as $$ F^TF = U\Sigma_F U^\dagger$$ $$GG^T = V\Sigma_G V^\dagger,$$ where $U,V$ are unitary matrices and $\Sigma_F,\Sigma_G$ are diagonal matrices with eigenvalues (non-negative) of $F^TF$ and $GG^T$ respectively as diagonal entries. Again, by using the cyclic property of trace function, we can write the left hand side as $$\mathsf{Tr}(F^TFGG^T) = \mathsf{Tr}(U\Sigma_F U^\dagger V\Sigma_G V^\dagger) = \mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V\Sigma_G ).$$

It is easy to show the following properties of diagonal matrices: Let $D$ be a diagonal matrix with non-negative diagonal entries.

1) Under any unitary transformation of $D$, the resulting matrix has non-negative diagonal entries.

2) If $M$ is a square matrix with non-negative diagonal entries, then $\mathsf{Tr}(MD)\leq \mathsf{Tr}(M) \cdot \mathsf{Tr} (D)$.

The above properties directly imply that $$\mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V\Sigma_G ) \leq \mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V) \cdot \mathsf{Tr}( \Sigma_G ) =\mathsf{Tr}( \Sigma_F ) \cdot \mathsf{Tr}( \Sigma_G ) = \mathsf{Tr}(F^TF) \mathsf{Tr}(GG^T),$$ where the last two equalities follow from the fact that trace is preserved under unitary transformation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.