I am having trouble with understanding the following definition while studying some basic things related with matrix norms:

For every matrix $A\in M_n(\mathbb{R})$

$$\sup_{x\neq 0}{} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1}{\|Ax\|},\; x \in\mathbb{R}^n$$

Why we are taking $\|x\| = 1$? Is there any proof of above statement?

Added: Why to take sup in the definition of norm?

I need help to understand above mentioned fact.

Thank you very much.

  • 4
    $\begingroup$ Notice that $Ax/\| x \| = A(x/\|x\|)$, and $x/\|x\|$ has norm 1, so it's equivalent to take the supremum over norm 1 vectors. $\endgroup$
    – Suugaku
    May 16 '13 at 18:22
  • $\begingroup$ It's also the sup of $\|Ax\|$ over $\|x\|\leq 1$. $\endgroup$
    – Julien
    May 17 '13 at 12:14

$$\frac{ \|Ax\| }{\|x\|} = \left|\left| A \frac{x}{\|x\|} \right|\right|$$ Running through all $x \neq 0$ is equivalent to running through all $y:= \frac{x}{\|x\|} $ with $\| y \|=1$.

  • 3
    $\begingroup$ Thanks for the answer. I was expecting little more explanation. Like why to take $\sup$? $\endgroup$
    – monalisa
    May 16 '13 at 18:32
  • 2
    $\begingroup$ @monalisa Because this definition is equivalent with definition $\| A \| = \inf_M \{ M : \| Ax \| \leqslant M \| x \| \}$, when $A$ is linear operator. $\endgroup$
    – Cortizol
    May 16 '13 at 19:03

As julien pointed out, what follows is not a proof that


it is a discussion about the consequences of $(1)$. For a proof of $(1)$ see MichaelNgelo's answer.

$(1)$ implies that if we know that a given vector achieves either of the two supremums in $(1)$, then we can deduce from it a vector that achieves the other supremum.

Specifically, $x^*$ is such that


if and only if $y^*:=\frac{x^*}{||x^*||}$ is such that


The above is useful because it allows to compute one of the supremums by computing the other instead, in particular, we can substitute a supremum over $\mathbb{R}\backslash \{0\}$ with one over the unit sphere.

An easy way to argue the above is by contradiction. For example, one direction follows from:

Suppose that $x^*$ satisfies $(*)$, but $y^*:=\frac{x^*}{||x^*||}$ does not satisfy $(**)$. Then, there exists some $\hat{y}\neq y^*$ such that $||\hat{y}||=1$ and

$$||A\hat{y}||>||Ay^*||\Rightarrow \frac{||A\hat{y}||}{1}=\frac{||A\hat{y}||}{||\hat{y}||}>||Ay^*||=\left|\left|A\frac{x^*}{||x^*||}\right|\right|=\frac{||Ax^*||}{||x^*||}=\sup_{x\neq0}\frac{||Ax||}{||x||}$$

which gives a contradiction.

EDIT: The $\sup$ is taken in order to make the function $\sigma$, that maps from the vector space of real $n\times n$ matrices, $M_n(\mathbb{R})$, to $[0,\infty)$, and is defined as

$$\sigma(A):=\sup_{x\neq 0}\frac{||Ax||}{||x||},$$

be a norm (that is, satisfy the norm axioms) on $M_n(\mathbb{R})$. If we drop the sup and instead use a fixed $x$, then $\sigma$ doesn't satisfy the first axiom ($\sigma(A)=0\Leftrightarrow A=0)$ any more.

  • 2
    $\begingroup$ @srijan thanks! $\endgroup$
    – jkn
    May 16 '13 at 19:49
  • $\begingroup$ nice explanation @jkn $\endgroup$
    – monalisa
    May 17 '13 at 4:07
  • $\begingroup$ glad its of any use. $\endgroup$
    – jkn
    May 17 '13 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.