# Frobenius norm of product of matrix

The Frobenius norm of a $m \times n$ matrix $F$ is defined as

$$\| F \|_F^2 := \sum_{i=1}^m\sum_{j=1}^n |f_{i,j}|^2$$

If I have $FG$, where $G$ is a $n \times p$ matrix, can we say the following?

$$\| F G \|_F^2 = \|F\|_F^2 \|G\|_F^2$$

Also, what does Frobenius norm mean? Is it analogous to the magnitude of a vector, but for matrix?

• in general, the Frobenius norm need not be multiplicative. it is just sub multiplicative. Think about Cauchy Schwarz inequality. – user251257 Aug 11 '15 at 15:49
• concerning the "meaning" of the norm, the set of matrices is a vector space which means up to a choice of notation it's just $\mathbb{R}^p$ for $p=n^2$. This Frobenius norm is just the natural length of the vector which is formed by stringing out the entries of the matrix into an $n^2$-vector. Furthermore, the norm makes the set of matrices a normed linear space which means you get all the excellent theorems which go with that structure. In particular, a nice theory of power series etc. – James S. Cook Aug 12 '15 at 1:57
• If we had equality in all cases then we'd have a normed algebra, but, that is too greedy except in a few special cases like the matrices which correspond to the reals, complex or quaternions. In other cases, the existence of zero-divisors in the algebra necessarily either spoils multiplicativity of the norm or it gives a multiplicative "norm" which isn't really a norm. – James S. Cook Aug 12 '15 at 2:01

Actually there is $$||FG||^2_F \leqslant||F||^2_F||G||^2_F$$ The proof is like: \begin{align} \|FG\|^2_F&=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left|\sum\limits_{k=1}^nf_{ik}g_{kj}\right|^2 \\ &\leqslant\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k=1}^n|f_{ik}|^2\sum\limits_{k=1}^n|g_{kj}|^2\right)\tag{Cauchy-Schwarz} \\ &=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{p}\left(\sum\limits_{k,l=1}^n|f_{ik}|^2|g_{lj}|^2\right) \\ &=\sum\limits_{i=1}^{m}\sum\limits_{k=1}^{n}|f_{ik}|^2\sum\limits_{l=1}^{n}\sum\limits_{j=1}^{p}|g_{lj}|^2 \\ &=\|F\|^2_F\|G\|^2_F \end{align} Frobenius norm like vector norm and similar to $$l_2$$.

• I wonder, in general, for which matrices is equality attained. – James S. Cook Aug 12 '15 at 0:55
• When Cauchy-Schwarz becomes equal. – Math Wizard Aug 12 '15 at 1:01
• Yes, how can we otherwise characterize such matrices... (your answer is appropriate to the question asked I think as the OP wants a general identity). – James S. Cook Aug 12 '15 at 1:25
• I could ask a stand-alone question so this one does not get further cluttered. – James S. Cook Aug 12 '15 at 1:50
• I mean I could ask a separate question about equality. Of course, you could just add a second answer here. – James S. Cook Aug 12 '15 at 2:10

Let $$A$$ be $$m \times r$$ and $$B$$ be $$r \times n$$. A better bound here is $$\| A B \|_F \le \|A\| \|B\|_F \quad (*)$$ where $$\|A\|$$ is the $$\ell_2$$ operator norm: $$\|A\| = \max_{\|x\|_2\, \le\, 1} \|A x\|_2.$$ It is also equal to the largest singular value of A. From this definition $$\|A x\|_2 \le \|A\| \|x\|_2$$ for any vector $$x$$ in $$\mathbb R^r.$$

Since $$\|A\| \le \|A\|_F$$, inequality (*) is a strictly better inequality than the sub-multiplicative inequality for the Frobenius norm.

To see the inequality, let $$B = [b_1 \mid b_2 \mid \cdots \mid b_n]$$ be the column decomposition of $$B$$. Then, $$A B = [Ab_1 \mid A b_2 \mid \dots \mid Ab_n]$$ is the column decomposition of $$AB$$. It follows that \begin{align*} \| A B \|_F^2 \le \sum_{j=1}^n \|A b_j\|^2 \le \|A\|^2 \sum_j \|b_j\|^2 = \|A\|^2 \|B\|_F^2. \end{align*}

We will prove that $$\Vert FG \Vert_f^2 \leq \Vert F \Vert_f^2 \cdot \Vert G \Vert_f^2.$$ We have $$\Vert FG \Vert_f^2 = \mathsf{Tr}(FG G^TF^T) = \mathsf{Tr} (F^TFGG^T),$$ by the cyclic property of trace function. To prove the theorem, it is enough if we show that $$\mathsf{Tr}(F^TFGG^T) \leq \mathsf{Tr}(F^TF) \mathsf{Tr}(GG^T).$$ Observe that $$F^TF$$ and $$GG^T$$ are positive semidefinite and symmetric matrices. Hence, they are diagonalizable and can be written as $$F^TF = U\Sigma_F U^\dagger$$ $$GG^T = V\Sigma_G V^\dagger,$$ where $$U,V$$ are unitary matrices and $$\Sigma_F,\Sigma_G$$ are diagonal matrices with eigenvalues (non-negative) of $$F^TF$$ and $$GG^T$$ respectively as diagonal entries. Again, by using the cyclic property of trace function, we can write the left hand side as $$\mathsf{Tr}(F^TFGG^T) = \mathsf{Tr}(U\Sigma_F U^\dagger V\Sigma_G V^\dagger) = \mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V\Sigma_G ).$$

It is easy to show the following properties of diagonal matrices: Let $$D$$ be a diagonal matrix with non-negative diagonal entries.

1) Under any unitary transformation of $$D$$, the resulting matrix has non-negative diagonal entries.

2) If $$M$$ is a square matrix with non-negative diagonal entries, then $$\mathsf{Tr}(MD)\leq \mathsf{Tr}(M) \cdot \mathsf{Tr} (D)$$.

The above properties directly imply that $$\mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V\Sigma_G ) \leq \mathsf{Tr}(V^\dagger U\Sigma_F U^\dagger V) \cdot \mathsf{Tr}( \Sigma_G ) =\mathsf{Tr}( \Sigma_F ) \cdot \mathsf{Tr}( \Sigma_G ) = \mathsf{Tr}(F^TF) \mathsf{Tr}(GG^T),$$ where the last two equalities follow from the fact that trace is preserved under unitary transformation.