5
$\begingroup$

The Euclidean distance between points p and q is the length of the line segment connecting them ($\overline{\mathbf{p}\mathbf{q}}$).

$$\begin{aligned}d(\mathbf {p} ,\mathbf {q} )=d(\mathbf {q} ,\mathbf {p} )&={\sqrt {(q_{1}-p_{1})^{2}+(q_{2}-p_{2})^{2}+\cdots +(q_{n}-p_{n})^{2}}}\\[8pt]&={\sqrt {\sum _{i=1}^{n}(q_{i}-p_{i})^{2}}}.\end{aligned} $$

The Euclidean norm, or Euclidean length, or magnitude of a vector measures the length of the vector

$$\left\|{\mathbf {p}}\right\|={\sqrt {p_{1}^{2}+p_{2}^{2}+\cdots +p_{n}^{2}}}={\sqrt {{\mathbf {p}}\cdot {\mathbf {p}}}},$$

similar to above, is there a geometric interpretation about the euclidean distance between of 2 matrices?

take this concrete example.

X = [[0, 1],
     [2, 3]]

Y = [[1, 2],
     [3, 4]]

what does this quantity represent?

euclidean_distances(X , Y)

array([[1.41421356, 4.24264069],
       [1.41421356, 1.41421356]])
$\endgroup$
4
  • $\begingroup$ Your concrete example doesn't make sense to me: the distance between two things is never a matrix, no matter whether the two things are vectors, matrices, sets, or anything else. What software did you use to compute it, and have you checked its documentation for the euclidean_distances function? $\endgroup$
    – user856
    Aug 23, 2019 at 3:43
  • 1
    $\begingroup$ Actually, it looks like it is just computing all the pairwise distances between the rows of $X$ and the rows of $Y$. Voting to close as this is a confusion about software rather than a question about mathematics. $\endgroup$
    – user856
    Aug 23, 2019 at 3:45
  • $\begingroup$ @Rahul, questions about mathematical software are explicitly on topic here. $\endgroup$ Aug 23, 2019 at 10:07
  • $\begingroup$ How do you go from 2 vectors to 2 matrices? I don't get it. $\endgroup$ Aug 25, 2019 at 1:54

2 Answers 2

5
$\begingroup$

A way to write the Frobenius norm of a $m\times n$ matrix $A$ is

$$\lVert A \rVert_F = \sqrt{\sum_{i=1}^n\sum_{j=1}^n |a_{ij}|^2} = \sqrt{\sum_{k}^{\min\{m,n\}} \sigma^2_k(A)},$$

where $\sigma^2_k(A)$ are the singular values of $A$. These are defined as the square roots of the non-negative eigenvalues of $A^\ast A$; in the case where $A$ is a matrix with real entries, there is indeed a geometrical interpretation: the set $\{Ax ~:~ |x|_2=1 \}\subset \mathbb R^n$ is an ellipsoid, and the lengths of its semi-axes are the singular values of $A$. This is a consequence of the singular value decomposition theorem, which lets us decompose $A$ into a "scaling" transformation (i.e. a diagonal matrix) conjugated with two "rotations" (i.e. unitary matrices).

Thus, given $x\in \mathbb R^n$, we can say that $\lVert A \rVert_F$ is a rough measure of how long is the vector $Ax$. By this logic, $\lVert A-B \rVert_F$ (for $A,B$ matrices of the same size) should give us a rough measure of how far are the images of the same vector $x$ under $A$ and $B$.


When talking matrix norms, a related concept is that of induced norms — and it is perhaps better suited to describe how "close" are the images of two matrices. The matrix norm induced by the Euclidean norm is defined as

$$\lVert A \rVert_2 = \sup_{|x|\neq 0}\frac{|Ax|_2}{|x|_2} = \sup_{|x|=1} |Ax|_2,$$

or equivalently $\lVert A \rVert_2 = \sqrt{\lambda}$ where $\lambda$ is the largest eigenvalue of $A^\ast A$ (and here $A^\ast$ is the conjugate, or Hermitian transpose). When $A$ is a square matrix in $M_{n\times n}(\mathbb C)$, it is often called the spectral radius of $A$, as by the previous definition every unit vector evaluated in $A$ is then contained in a ball of radius $\lVert A \rVert_2$, i.e.

$$|Ax|_2 \leq \lVert A \rVert_2 ~~~~~\text{for all $x\in\mathbb C^n$.}$$

Now if $A,B$ are matrices in $M_{n\times n}$, what does the difference $\lVert A-B \rVert_2$ mean, geometrically? For arbitrary $x \in \mathbb C$, we have:

$$\lvert (A-B)x \rvert_2 = \lvert Ax-Bx \rvert_2 \leq \lVert A-B \rVert_2 |x|_2$$

That is: the distance between the image vectors $Ax$ and $Bx$ is at most $\lVert A-B \rVert_2 |x|_2$. If either $\lVert A-B \rVert_2$ or $|x|_2$ are small, you can expect the images to be very close.

$\endgroup$
3
  • $\begingroup$ As a small bonus, here's an actual research article (about non-autonomous linear ODEs) where such a norm $\lVert A-B \rVert$ is studied. In page 19, the author defines a homogeneous system of non-autonomous linear ODEs $x'=A(t)x$ (where $A(t)$ can be regarded as a "matrix function" or more generally a linear operator between Banach spaces) to be structurally stable if there exists $\delta>0$ such that if $\lVert B(t)-A(t) \rVert < \delta$, then there exists a homeomorphism between the solutions of $x'=A(t)x$ and $y'=B(t)y$. So, good question! $\endgroup$
    – B. Núñez
    Aug 23, 2019 at 3:14
  • $\begingroup$ The first sum should be from $1$ to $m$, shouldn't it? $\endgroup$
    – David K
    Aug 23, 2019 at 12:28
  • $\begingroup$ Ah, yes. I'll fix the typo. $\endgroup$
    – B. Núñez
    Aug 25, 2019 at 1:02
1
$\begingroup$

The Euclidean distance between two matrices, the square-root of the sum of the differences of the individual components, is called the Frobenius norm.

The set of $M\times N$ matrices of real numbers (together with point-wise addition and scalar multiplication) is a vector space! That means that a matrix is actually a vector (in a slightly more abstract sense). Therefore, the geometric intuition that you have with high-dimensional vectors holds with matrices.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .