The Frobenius norm of a $m \times n$ matrix $F$ is defined as

$$\| F \|_F^2 := \sum_{i=1}^m\sum_{j=1}^n |f_{i,j}|^2$$

If I have $FG$, where $G$ is a $n \times p$ matrix, can we say the following?

$$\| F G \|_F^2 = \|F\|_F^2 \|G\|_F^2$$

Also, what does Frobenius norm mean? Is it analogous to the magnitude of a vector, but for matrix?

  • 3
    $\begingroup$ in general, the Frobenius norm need not be multiplicative. it is just sub multiplicative. Think about Cauchy Schwarz inequality. $\endgroup$
    – user251257
    Aug 11 '15 at 15:49
  • $\begingroup$ concerning the "meaning" of the norm, the set of matrices is a vector space which means up to a choice of notation it's just $\mathbb{R}^p$ for $p=n^2$. This Frobenius norm is just the natural length of the vector which is formed by stringing out the entries of the matrix into an $n^2$-vector. Furthermore, the norm makes the set of matrices a normed linear space which means you get all the excellent theorems which go with that structure. In particular, a nice theory of power series etc. $\endgroup$ Aug 12 '15 at 1:57
  • $\begingroup$ If we had equality in all cases then we'd have a normed algebra, but, that is too greedy except in a few special cases like the matrices which correspond to the reals, complex or quaternions. In other cases, the existence of zero-divisors in the algebra necessarily either spoils multiplicativity of the norm or it gives a multiplicative "norm" which isn't really a norm. $\endgroup$ Aug 12 '15 at 2:01

Actually there is $$||FG||^2_F \leqslant||F||^2_F||G||^2_F$$ The proof is as follows. \begin{align} \|FG\|^2_F&=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left|\sum\limits_{k=1}^nf_{ik}g_{kj}\right|^2 \\ &\leqslant\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k=1}^n|f_{ik}|^2\sum\limits_{k=1}^n|g_{kj}|^2\right)\tag{Cauchy-Schwarz} \\ &=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k,l=1}^n|f_{ik}|^2|g_{lj}|^2\right) \\ &=\sum\limits_{i=1}^{m}\sum\limits_{k=1}^{n}|f_{ik}|^2\sum\limits_{l=1}^{n}\sum\limits_{j=1}^{p}|g_{lj}|^2 \\ &=\|F\|^2_F\|G\|^2_F \end{align} Frobenius norm is like vector norm and similar to $l_2$.

  • $\begingroup$ I wonder, in general, for which matrices is equality attained. $\endgroup$ Aug 12 '15 at 0:55
  • 1
    $\begingroup$ When Cauchy-Schwarz becomes equal. $\endgroup$
    – hermes
    Aug 12 '15 at 1:01
  • $\begingroup$ Yes, how can we otherwise characterize such matrices... (your answer is appropriate to the question asked I think as the OP wants a general identity). $\endgroup$ Aug 12 '15 at 1:25
  • $\begingroup$ I could ask a stand-alone question so this one does not get further cluttered. $\endgroup$ Aug 12 '15 at 1:50
  • 3
    $\begingroup$ The equality discussion of Cauchy-Schwarz yields that we have equality if and only if $F = x \otimes y^t$, $G = y \otimes z^t$ for vectors $x, y, z$. In particular, $F$ and $G$ have rank one. $\endgroup$
    – Kofi
    Sep 4 '19 at 23:20

Let $A$ be $m \times r$ and $B$ be $r \times n$. A better bound here is $$ \| A B \|_F \le \|A\| \|B\|_F \quad (*) $$ where $\|A\|$ is the $\ell_2$ operator norm: $$ \|A\| = \max_{\|x\|_2\, \le\, 1} \|A x\|_2. $$ It is also equal to the largest singular value of A. From this definition $\|A x\|_2 \le \|A\| \|x\|_2$ for any vector $x$ in $\mathbb R^r.$

Since $\|A\| \le \|A\|_F$, inequality (*) is a strictly better inequality than the sub-multiplicative inequality for the Frobenius norm.

To see the inequality, let $B = [b_1 \mid b_2 \mid \cdots \mid b_n]$ be the column decomposition of $B$. Then, $A B = [Ab_1 \mid A b_2 \mid \dots \mid Ab_n]$ is the column decomposition of $AB$. It follows that \begin{align*} \| A B \|_F^2 = \sum_{j=1}^n \|A b_j\|^2 \le \|A\|^2 \sum_j \|b_j\|^2 = \|A\|^2 \|B\|_F^2. \end{align*}

EDIT in response to the question in the comments, ``Is there a lower bound for the Frobenius norm of the product of two matrices?''. In general, no, except for the obvious lower bound of zero. Consider the following two matrices \begin{align} A = \begin{pmatrix} a & b \\ 0 & 0 \end{pmatrix}, \quad B = \begin{pmatrix} -b & 0 \\ a & 0 \end{pmatrix}. \end{align} Then $\|A\|_F = \|B\|_F = \sqrt{a^2 + b^2}$, while $\|AB\|_F = 0$.

What if the two matrices are symmetric? Consider \begin{align} A = \begin{pmatrix} a & b \\ b & a \end{pmatrix}, \quad B = \begin{pmatrix} -b & a \\ a & -b \end{pmatrix}, \quad A B = \begin{pmatrix} 0 & a^2-b^2 \\ a^2-b^2 & 0 \end{pmatrix}. \end{align} Then, $\|A\|_F^2 = \|B\|_F^2 = 2(a^2 + b^2)$ while $\|AB\|_F^2 = 2(a^2 - b^2)^2$ which can be made arbitrarily smaller than either of $\|A\|_F^2$ or $\|B\|_F^2$. For example, take $a=b$.

  • 1
    $\begingroup$ Nice answer, thanks! Is there a (perhaps sharp or non-trivial) lower bound for the Frobenius norm of the product of two matrices? Thanks! $\endgroup$ Mar 6 '20 at 10:34
  • $\begingroup$ @Mathmath, no. See my edit to the response. $\endgroup$
    – passerby51
    Mar 6 '20 at 18:06

We will prove that $$ \Vert FG \Vert_f^2 \leq \Vert F \Vert_f^2 \cdot \Vert G \Vert_f^2.$$ We have $$\Vert FG \Vert_f^2 = \mathsf{Tr}(FG G^TF^T) = \mathsf{Tr} (F^TFGG^T),$$ by the cyclic property of trace function. To prove the theorem, it is enough if we show that $$\mathsf{Tr}(F^TFGG^T) \leq \mathsf{Tr}(F^TF) \mathsf{Tr}(GG^T).$$ Observe that $F^TF$ and $GG^T$ are positive semidefinite and symmetric matrices. Hence, they are diagonalizable and can be written as $$ F^TF = U\Sigma_F U^\dagger$$ $$GG^T = V\Sigma_G V^\dagger,$$ where $U,V$ are unitary matrices and $\Sigma_F,\Sigma_G$ are diagonal matrices with eigenvalues (non-negative) of $F^TF$ and $GG^T$ respectively as diagonal entries. Again, by using the cyclic property of trace function, we can write the left hand side as $$\mathsf{Tr}(F^TFGG^T) = \mathsf{Tr}(U\Sigma_F U^\dagger V\Sigma_G V^\dagger) = \mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V\Sigma_G ).$$

It is easy to show the following properties of diagonal matrices: Let $D$ be a diagonal matrix with non-negative diagonal entries.

1) Under any unitary transformation of $D$, the resulting matrix has non-negative diagonal entries.

2) If $M$ is a square matrix with non-negative diagonal entries, then $\mathsf{Tr}(MD)\leq \mathsf{Tr}(M) \cdot \mathsf{Tr} (D)$.

The above properties directly imply that $$\mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V\Sigma_G ) \leq \mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V) \cdot \mathsf{Tr}( \Sigma_G ) =\mathsf{Tr}( \Sigma_F ) \cdot \mathsf{Tr}( \Sigma_G ) = \mathsf{Tr}(F^TF) \mathsf{Tr}(GG^T),$$ where the last two equalities follow from the fact that trace is preserved under unitary transformation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.