12
$\begingroup$

I am having trouble with understanding the following definition while studying some basic things related with matrix norms:

For every matrix $A\in M_n(\mathbb{R})$

$$\sup_{x\neq 0}{} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1}{\|Ax\|},\; x \in\mathbb{R}^n$$

Why we are taking $\|x\| = 1$? Is there any proof of above statement?

Added: Why to take sup in the definition of norm?

I need help to understand above mentioned fact.

Thank you very much.

$\endgroup$
2
  • 4
    $\begingroup$ Notice that $Ax/\| x \| = A(x/\|x\|)$, and $x/\|x\|$ has norm 1, so it's equivalent to take the supremum over norm 1 vectors. $\endgroup$
    – Suugaku
    Commented May 16, 2013 at 18:22
  • $\begingroup$ It's also the sup of $\|Ax\|$ over $\|x\|\leq 1$. $\endgroup$
    – Julien
    Commented May 17, 2013 at 12:14

2 Answers 2

11
$\begingroup$

$$\frac{ \|Ax\| }{\|x\|} = \left|\left| A \frac{x}{\|x\|} \right|\right|$$ Running through all $x \neq 0$ is equivalent to running through all $y:= \frac{x}{\|x\|} $ with $\| y \|=1$.

$\endgroup$
2
  • 3
    $\begingroup$ Thanks for the answer. I was expecting little more explanation. Like why to take $\sup$? $\endgroup$
    – monalisa
    Commented May 16, 2013 at 18:32
  • 2
    $\begingroup$ @monalisa Because this definition is equivalent with definition $\| A \| = \inf_M \{ M : \| Ax \| \leqslant M \| x \| \}$, when $A$ is linear operator. $\endgroup$
    – Cortizol
    Commented May 16, 2013 at 19:03
5
$\begingroup$

As julien pointed out, what follows is not a proof that

$$\sup_{x\neq0}\frac{||Ax||}{||x||}=\sup_{||y||=1}||Ay||,\quad\quad(1)$$

it is a discussion about the consequences of $(1)$. For a proof of $(1)$ see MichaelNgelo's answer.


$(1)$ implies that if we know that a given vector achieves either of the two supremums in $(1)$, then we can deduce from it a vector that achieves the other supremum.

Specifically, $x^*$ is such that

$$\frac{||Ax^*||}{||x^*||}=\sup_{x\neq0}\frac{||Ax||}{||x||}\quad\quad(*)$$

if and only if $y^*:=\frac{x^*}{||x^*||}$ is such that

$$||Ay^*||=\sup_{||y||=1}||Ay||.\quad\quad(**)$$

The above is useful because it allows to compute one of the supremums by computing the other instead, in particular, we can substitute a supremum over $\mathbb{R}\backslash \{0\}$ with one over the unit sphere.


An easy way to argue the above is by contradiction. For example, one direction follows from:

Suppose that $x^*$ satisfies $(*)$, but $y^*:=\frac{x^*}{||x^*||}$ does not satisfy $(**)$. Then, there exists some $\hat{y}\neq y^*$ such that $||\hat{y}||=1$ and

$$||A\hat{y}||>||Ay^*||\Rightarrow \frac{||A\hat{y}||}{1}=\frac{||A\hat{y}||}{||\hat{y}||}>||Ay^*||=\left|\left|A\frac{x^*}{||x^*||}\right|\right|=\frac{||Ax^*||}{||x^*||}=\sup_{x\neq0}\frac{||Ax||}{||x||}$$

which gives a contradiction.


EDIT: The $\sup$ is taken in order to make the function $\sigma$, that maps from the vector space of real $n\times n$ matrices, $M_n(\mathbb{R})$, to $[0,\infty)$, and is defined as

$$\sigma(A):=\sup_{x\neq 0}\frac{||Ax||}{||x||},$$

be a norm (that is, satisfy the norm axioms) on $M_n(\mathbb{R})$. If we drop the sup and instead use a fixed $x$, then $\sigma$ doesn't satisfy the first axiom ($\sigma(A)=0\Leftrightarrow A=0)$ any more.

$\endgroup$
3
  • 2
    $\begingroup$ @srijan thanks! $\endgroup$
    – jkn
    Commented May 16, 2013 at 19:49
  • $\begingroup$ nice explanation @jkn $\endgroup$
    – monalisa
    Commented May 17, 2013 at 4:07
  • $\begingroup$ glad its of any use. $\endgroup$
    – jkn
    Commented May 17, 2013 at 12:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .