71
$\begingroup$

I'm in the process of writing an application which identifies the closest matrix from a set of square matrices $M$ to a given square matrix $A$. The closest can be defined as the most similar.

I think finding the distance between two given matrices is a fair approach since the smallest Euclidean distance is used to identify the closeness of vectors.

I found that the distance between two matrices ($A,B$) could be calculated using the Frobenius distance $F$:

$$F_{A,B} = \sqrt{trace((A-B)*(A-B)')} $$

where $B'$ represents the conjugate transpose of B.

I have the following points I need to clarify

  • Is the distance between matrices a fair measure of similarity?
  • If distance is used, is Frobenius distance a fair measure for this problem? any other suggestions?
$\endgroup$
3
  • 4
    $\begingroup$ There are many ways to measure the "distance" between two matrices (just as there are many ways to measure the distance between two vectors). Without some more information, it's impossible to say which one is best for you. $\endgroup$
    – bubba
    Sep 28, 2013 at 12:40
  • $\begingroup$ @bubba I just want to find the closest matrix to a give matrix numerically. I'm creating a closest match retriever for a given matrix. If you can let me know the other possible methods you know for distance measures that would be a great help. $\endgroup$
    – Synex
    Sep 28, 2013 at 15:51
  • 3
    $\begingroup$ Provided that you "just want to compute something", a good norm is the norm for which the problem can be easily solved. This is often the case for the Frobenius norm. $\endgroup$ Sep 29, 2013 at 11:46

3 Answers 3

62
$\begingroup$

Some suggestions. Too long for a comment:

As I said, there are many ways to measure the "distance" between two matrices. If the matrices are $\mathbf{A} = (a_{ij})$ and $\mathbf{B} = (b_{ij})$, then some examples are: $$ d_1(\mathbf{A}, \mathbf{B}) = \sum_{i=1}^n \sum_{j=1}^n |a_{ij} - b_{ij}| $$ $$ d_2(\mathbf{A}, \mathbf{B}) = \sqrt{\sum_{i=1}^n \sum_{j=1}^n (a_{ij} - b_{ij})^2} $$ $$ d_\infty(\mathbf{A}, \mathbf{B}) = \max_{1 \le i \le n}\max_{1 \le j \le n} |a_{ij} - b_{ij}| $$ $$ d_m(\mathbf{A}, \mathbf{B}) = \max\{ \|(\mathbf{A} - \mathbf{B})\mathbf{x}\| : \mathbf{x} \in \mathbb{R}^n, \|\mathbf{x}\| = 1 \} $$ I'm sure there are many others. If you look up "matrix norms", you'll find lots of material. And if $\|\;\|$ is any matrix norm, then $\| \mathbf{A} - \mathbf{B}\|$ gives you a measure of the "distance" between two matrices $\mathbf{A}$ and $\mathbf{B}$.

Or, you could simply count the number of positions where $|a_{ij} - b_{ij}|$ is larger than some threshold number. This doesn't have all the nice properties of a distance derived from a norm, but it still might be suitable for your needs.

These distance measures all have somewhat different properties. For example, the third one shown above will tell you that two matrices are far apart even if all their entries are the same except for a large difference in one position.

$\endgroup$
5
  • 1
    $\begingroup$ Do all of these satisfy the conditions of a metric space? $\endgroup$
    – eric
    Aug 17, 2018 at 15:32
  • 2
    $\begingroup$ The ones for which I gave formulas are distances derived from matrix norms, so they are distances in the metric space sense. The last one (counting large differences) is not a metric. $\endgroup$
    – bubba
    Aug 18, 2018 at 2:28
  • $\begingroup$ Can you please provide the references for these formulas? $\endgroup$
    – Shayan
    Mar 27, 2022 at 12:11
  • $\begingroup$ As I said, look up “matrix norms” $\endgroup$
    – bubba
    Mar 27, 2022 at 22:31
  • $\begingroup$ OP mentioned square matrices, so if they have the additional property of being covariance matrices, then I suggest looking into: W. Förstner and B. Moonen, A Metric for Covariance Matrices, 1999 (and 2003). $\endgroup$
    – Number
    Dec 30, 2023 at 0:44
10
$\begingroup$

If we have two matrices $A,B$. Distance between $A$ and $B$ can be calculated using Singular values or $2$ norms.

You may use Distance $= \vert(\text{fnorm}(A)-\text{fnorm}(B))\vert$ where fnorm = sq root of sum of squares of all singular values.

$\endgroup$
2
  • 1
    $\begingroup$ Can you expand some on this answer? What are the properties of this distance / where did you get it from? $\endgroup$ Oct 6, 2020 at 22:13
  • $\begingroup$ I think this is not always correct, you can find two matrices that are quite different but they have the same singular values math.stackexchange.com/questions/4191889/… $\endgroup$ Jul 7, 2021 at 10:11
4
$\begingroup$

A possible measure of similarity between matrices is: $$ D(AB)=\sum_{i,j}A_{ij}\left[\log\frac{A_{ij}}{B_{ij}} + \log B - \log A\right], \text{ where } A=\sum_{i,j}A_{ij}, B=\sum_{i,j}B_{ij}. $$ This measure is inspired by Kullback-Leibler divergence for probability vectors $$ p_{ij}=\frac{A_{ij}}{A},q_{ij}=\frac{B_{ij}}{B}. $$ Note that this is not a distance, as it is not symmetric in respect to $A$ and $B$, however, it could be symmetrized, e.g., as Jensen-Shannon divergence. It is quite useful as is, e.g., when one measures divergence from the ideal/exact result.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .