Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a positive definite matrix $A$. I am going to choose its Frobenius norm $\|A\|_F^2$ as a cost function and then minimize $\|A\|_F^2$. But I think I need to find a reason to convince people it is reasonable to choose $\|A\|_F^2$ as a cost function. So I'm wondering if there are some geometric meanings of the Frobenius norm. Thanks.

Edit: here $A$ is a 3 by 3 matrix. In the problem I'm working on, people usually choose $\det A$ as a cost function since $\det A$ has an obvious geometric interpretation: the volume of the parallelepiped determined by $A$. Now I want to choose $\|A\|_F^2$ as a cost function because of the good properties of $\|A\|_F^2$. That's why I am interested in the geometric meaning of $\|A\|_F^2$.

share|improve this question
    
$\|A\|_F^2$ is equal to summation of the squre of all singular values of $A$. Perhaps you could start from the geometric meaning of singular values of a matrix. –  Sunni Jul 28 '11 at 1:34
    
Since all matrix norms are equivaluent, it doesn't matter which norm you choose as a cost function. –  Sunni Jul 28 '11 at 1:35
    
@Sunni: For the problem I'm working on, people usually choose $\det A$ as the cost function as $\det A$ has an obvious geometric meaning: the volume of the parallelepiped determined by $A$. I know $\|A\|_F^2=trace (AA^T)=\sum\sigma_i^2$. But what is the geometric meaning of the sum of singular values? –  Shiyu Jul 28 '11 at 2:03

1 Answer 1

up vote 9 down vote accepted

To give intuition, lets look at a particularly nice class of matrices. Suppose the matrix $A$ is symmetric. Then we can find a basis of $n$ orthonormal eigenvectors $v_1,\cdots v_n$ with eigenvalues $\lambda_1,\dots,\lambda_n$. Each eigenvalue geometrically represents the "stretching factor" in the direction of its associated eigenvector.

The Frobenius norm of $A$ in this case is $$\|A\|_F =\sqrt{\text{Tr}(AA^t)}=\sqrt{\text{Tr}(A^2)}=\sqrt{\lambda_1^2+\cdots+\lambda_n^2}.$$ Working in the basis $v_1,\cdots v_n$, consider the box which is the image of the unit cube under $A$. The Frobenius norm is the diagonal of that box, and the determinant is the volume.

The usual norm defined as $\sup_{\|x\|=1}\|Ax\|$ corresponds to the longest side of the box.

share|improve this answer
    
Very interesting. If not working in the basis $v_1,...,v_n$, $A$ is corresponding a parallelepiped. Then $\det A$ is still the volume of the parallelepiped, can we say $\|A\|_F$ is still the diagonal of the parallelepiped? –  Shiyu Jul 28 '11 at 2:24
    
@Shiyu: By SVD, yes, but it will be the diagonal of a parallelepiped that's of smaller dimension than the ambient space. –  anon Jul 28 '11 at 3:27
    
@Eric this is quite nice explanation, however I am not sure what you mean by "consider the box which is the image of the unit cube under A"... What? :-) I think I know what you mean... let us say you have a skewed box, so the Fro-norm is the longest diagonal within the box that touches the corners, while the determinant is the volume of this box? –  Mohammad Aug 30 '12 at 15:25
    
@Mohammad: Well it is more like an $n$-dimensional parallelogram than a box if that helps. –  Eric Naslund Aug 30 '12 at 16:57

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.