Is there a measure for matrix that is analogous to rank of the matrix, but it is continuous on matrix elements? Say, we could say the information in identity matrix $I_n$ is $n$, and when the off-diagonal elements change from 0 to 1, the information contained in the matrix reduces continuously.

Example: considering a matrix $A(a)$ \begin{pmatrix} 1&0&0&\\ 0&1&a\\ 0&a&1 \end{pmatrix}. How do I define an information measure $I(A(a))$ that is continuous to $a$ for matrix $A(a)$, so that $I(A(0)) = I(I_3) = 3$ and $I(A(1)) = I(I_2)= 2$?

Rank is not continuous; Shannon information entropy on eigenvalues does not give desired values; and von Neumann entropy $S(\rho) = -tr(\rho\log(\rho))$ equals zero for any identity matrix while the dimension information lost; and $S(A(1))$ from my example does not reduce to S($I_2$).


One way is the nuclear norm, which is the sum of singular values.

It's often used in low rank matrix completion, such as in "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization" by B. Recht et al.

  • $\begingroup$ sum signular values of $A(1)$ is 3,(2+1+0) same as for $A(0)$ 3 = 1 + 1 +1? $\endgroup$ – ahala Jun 16 '16 at 18:07
  • $\begingroup$ Its a commonly used surrogate for rank which is continuous and even convex, but will not necessarily give you the same result as rank (even on your example). $\endgroup$ – Batman Jun 16 '16 at 18:10

This seems close: $$ I(A(a)) = \frac{n^2}{||A(a)||^2_F}$$, where $||A(a)||_F$ is the matrix entrywise 2-norm. For the example in the question: $ I(A(0)) = 3$ and $ I(A(1)) = 9/5 = 1.8$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.