What is the Euclidean norm of a M × N matrices?

Question

We define the euclidean norm as following: $$ \lvert x \rvert= (\sum_{j=1}^{N}(\lvert x_j \rvert)^2)^{1/2}=(x.x)^{1/2} $$

Assume $A$ is a $M$x$N$ matrices is that true: $\lvert Ax \rvert=\lvert A \rvert\lvert x \rvert$ ? or how can we write $\lvert Ax \rvert$ ?

Thanks in advance for any help!

Answer 1

There are various ways of defining the norm for a matrix. Here is the most common:

$$||A|| := \sup\limits_{||x|| = 1} ||Ax|| = \sup_x \dfrac{||Ax||}{||x||}$$

(Here, it is assumed that you choose the Euclidean $2$-norm for $||x||$, but you could very well choose another norm on your vector space, and get another norm for your matrices).

So then, you do not necessarily have $||Ax|| = ||A||\cdot||x||$, because since $||A||$ is defined as the supremum of $\dfrac{||Ax||}{||x||}$, then for arbitrary $x$, we have $\dfrac{||Ax||}{||x||} \leq ||A|| \implies ||Ax|| \leq ||A||\cdot||x||$.

---

Just for fun, here is another very used norm:

$$||A||_\infty := \sup_{i,j} |a_{ij}|$$Where the $a_{ij}$ are the coefficients of the matrix $A$.

Fortunately, there is a theorem that tells us that in a finite-dimensional vector space (so, for example, the space of all $M\times N$ matrices), all norms are equivalent, in the sense that if you have two norms $||\cdot||_1, ||\cdot||_2$, then there exist positive constants $c_1 < c_2$ such that for every element $x$ of your vector space, $c_1||x||_1 \leq ||x||_2 \leq c_2||x||_2$. This tells us that nothing really gets out of hand, and if a theorem holds with one norm, it very likely does for the others. Here is a nice pdf on it.