5
$\begingroup$

The Euclidean distance between points p and q is the length of the line segment connecting them ($\overline{\mathbf{p}\mathbf{q}}$).

$$\begin{aligned}d(\mathbf {p} ,\mathbf {q} )=d(\mathbf {q} ,\mathbf {p} )&={\sqrt {(q_{1}-p_{1})^{2}+(q_{2}-p_{2})^{2}+\cdots +(q_{n}-p_{n})^{2}}}\\[8pt]&={\sqrt {\sum _{i=1}^{n}(q_{i}-p_{i})^{2}}}.\end{aligned} $$

The Euclidean norm, or Euclidean length, or magnitude of a vector measures the length of the vector

$$\left\|{\mathbf {p}}\right\|={\sqrt {p_{1}^{2}+p_{2}^{2}+\cdots +p_{n}^{2}}}={\sqrt {{\mathbf {p}}\cdot {\mathbf {p}}}},$$

similar to above, is there a geometric interpretation about the euclidean distance between of 2 matrices?

take this concrete example.

X = [[0, 1],
     [2, 3]]

Y = [[1, 2],
     [3, 4]]

what does this quantity represent?

euclidean_distances(X , Y)

array([[1.41421356, 4.24264069],
       [1.41421356, 1.41421356]])
$\endgroup$
  • $\begingroup$ Your concrete example doesn't make sense to me: the distance between two things is never a matrix, no matter whether the two things are vectors, matrices, sets, or anything else. What software did you use to compute it, and have you checked its documentation for the euclidean_distances function? $\endgroup$ – user856 Aug 23 '19 at 3:43
  • 1
    $\begingroup$ Actually, it looks like it is just computing all the pairwise distances between the rows of $X$ and the rows of $Y$. Voting to close as this is a confusion about software rather than a question about mathematics. $\endgroup$ – user856 Aug 23 '19 at 3:45
  • $\begingroup$ @Rahul, questions about mathematical software are explicitly on topic here. $\endgroup$ – Mees de Vries Aug 23 '19 at 10:07
  • $\begingroup$ How do you go from 2 vectors to 2 matrices? I don't get it. $\endgroup$ – John Alexiou Aug 25 '19 at 1:54
4
$\begingroup$

A way to write the Frobenius norm of a $m\times n$ matrix $A$ is

$$\lVert A \rVert_F = \sqrt{\sum_{i=1}^n\sum_{j=1}^n |a_{ij}|^2} = \sqrt{\sum_{k}^{\min\{m,n\}} \sigma^2_k(A)},$$

where $\sigma^2_k(A)$ are the singular values of $A$. These are defined as the square roots of the non-negative eigenvalues of $A^\ast A$; in the case where $A$ is a matrix with real entries, there is indeed a geometrical interpretation: the set $\{Ax ~:~ |x|_2=1 \}\subset \mathbb R^n$ is an ellipsoid, and the lengths of its semi-axes are the singular values of $A$. This is a consequence of the singular value decomposition theorem, which lets us decompose $A$ into a "scaling" transformation (i.e. a diagonal matrix) conjugated with two "rotations" (i.e. unitary matrices).

Thus, given $x\in \mathbb R^n$, we can say that $\lVert A \rVert_F$ is a rough measure of how long is the vector $Ax$. By this logic, $\lVert A-B \rVert_F$ (for $A,B$ matrices of the same size) should give us a rough measure of how far are the images of the same vector $x$ under $A$ and $B$.


When talking matrix norms, a related concept is that of induced norms — and it is perhaps better suited to describe how "close" are the images of two matrices. The matrix norm induced by the Euclidean norm is defined as

$$\lVert A \rVert_2 = \sup_{|x|\neq 0}\frac{|Ax|_2}{|x|_2} = \sup_{|x|=1} |Ax|_2,$$

or equivalently $\lVert A \rVert_2 = \sqrt{\lambda}$ where $\lambda$ is the largest eigenvalue of $A^\ast A$ (and here $A^\ast$ is the conjugate, or Hermitian transpose). When $A$ is a square matrix in $M_{n\times n}(\mathbb C)$, it is often called the spectral radius of $A$, as by the previous definition every unit vector evaluated in $A$ is then contained in a ball of radius $\lVert A \rVert_2$, i.e.

$$|Ax|_2 \leq \lVert A \rVert_2 ~~~~~\text{for all $x\in\mathbb C^n$.}$$

Now if $A,B$ are matrices in $M_{n\times n}$, what does the difference $\lVert A-B \rVert_2$ mean, geometrically? For arbitrary $x \in \mathbb C$, we have:

$$\lvert (A-B)x \rvert_2 = \lvert Ax-Bx \rvert_2 \leq \lVert A-B \rVert_2 |x|_2$$

That is: the distance between the image vectors $Ax$ and $Bx$ is at most $\lVert A-B \rVert_2 |x|_2$. If either $\lVert A-B \rVert_2$ or $|x|_2$ are small, you can expect the images to be very close.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ As a small bonus, here's an actual research article (about non-autonomous linear ODEs) where such a norm $\lVert A-B \rVert$ is studied. In page 19, the author defines a homogeneous system of non-autonomous linear ODEs $x'=A(t)x$ (where $A(t)$ can be regarded as a "matrix function" or more generally a linear operator between Banach spaces) to be structurally stable if there exists $\delta>0$ such that if $\lVert B(t)-A(t) \rVert < \delta$, then there exists a homeomorphism between the solutions of $x'=A(t)x$ and $y'=B(t)y$. So, good question! $\endgroup$ – B. Núñez Aug 23 '19 at 3:14
  • $\begingroup$ The first sum should be from $1$ to $m$, shouldn't it? $\endgroup$ – David K Aug 23 '19 at 12:28
  • $\begingroup$ Ah, yes. I'll fix the typo. $\endgroup$ – B. Núñez Aug 25 '19 at 1:02
1
$\begingroup$

The Euclidean distance between two matrices, the square-root of the sum of the differences of the individual components, is called the Frobenius norm.

The set of $M\times N$ matrices of real numbers (together with point-wise addition and scalar multiplication) is a vector space! That means that a matrix is actually a vector (in a slightly more abstract sense). Therefore, the geometric intuition that you have with high-dimensional vectors holds with matrices.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.