# Basic question about $\sup_{x\neq 0}{} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1}{\|Ax\|}$, $x \in\mathbb{R}^n$

I am having trouble with understanding the following definition while studying some basic things related with matrix norms:

For every matrix $A\in M_n(\mathbb{R})$

$$\sup_{x\neq 0}{} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1}{\|Ax\|},\; x \in\mathbb{R}^n$$

Why we are taking $\|x\| = 1$? Is there any proof of above statement?

Added: Why to take sup in the definition of norm?

I need help to understand above mentioned fact.

Thank you very much.

• Notice that $Ax/\| x \| = A(x/\|x\|)$, and $x/\|x\|$ has norm 1, so it's equivalent to take the supremum over norm 1 vectors. Commented May 16, 2013 at 18:22
• It's also the sup of $\|Ax\|$ over $\|x\|\leq 1$. Commented May 17, 2013 at 12:14

$$\frac{ \|Ax\| }{\|x\|} = \left|\left| A \frac{x}{\|x\|} \right|\right|$$ Running through all $x \neq 0$ is equivalent to running through all $y:= \frac{x}{\|x\|}$ with $\| y \|=1$.

• Thanks for the answer. I was expecting little more explanation. Like why to take $\sup$? Commented May 16, 2013 at 18:32
• @monalisa Because this definition is equivalent with definition $\| A \| = \inf_M \{ M : \| Ax \| \leqslant M \| x \| \}$, when $A$ is linear operator. Commented May 16, 2013 at 19:03

As julien pointed out, what follows is not a proof that

$$\sup_{x\neq0}\frac{||Ax||}{||x||}=\sup_{||y||=1}||Ay||,\quad\quad(1)$$

it is a discussion about the consequences of $(1)$. For a proof of $(1)$ see MichaelNgelo's answer.

$(1)$ implies that if we know that a given vector achieves either of the two supremums in $(1)$, then we can deduce from it a vector that achieves the other supremum.

Specifically, $x^*$ is such that

$$\frac{||Ax^*||}{||x^*||}=\sup_{x\neq0}\frac{||Ax||}{||x||}\quad\quad(*)$$

if and only if $y^*:=\frac{x^*}{||x^*||}$ is such that

$$||Ay^*||=\sup_{||y||=1}||Ay||.\quad\quad(**)$$

The above is useful because it allows to compute one of the supremums by computing the other instead, in particular, we can substitute a supremum over $\mathbb{R}\backslash \{0\}$ with one over the unit sphere.

An easy way to argue the above is by contradiction. For example, one direction follows from:

Suppose that $x^*$ satisfies $(*)$, but $y^*:=\frac{x^*}{||x^*||}$ does not satisfy $(**)$. Then, there exists some $\hat{y}\neq y^*$ such that $||\hat{y}||=1$ and

$$||A\hat{y}||>||Ay^*||\Rightarrow \frac{||A\hat{y}||}{1}=\frac{||A\hat{y}||}{||\hat{y}||}>||Ay^*||=\left|\left|A\frac{x^*}{||x^*||}\right|\right|=\frac{||Ax^*||}{||x^*||}=\sup_{x\neq0}\frac{||Ax||}{||x||}$$

EDIT: The $\sup$ is taken in order to make the function $\sigma$, that maps from the vector space of real $n\times n$ matrices, $M_n(\mathbb{R})$, to $[0,\infty)$, and is defined as

$$\sigma(A):=\sup_{x\neq 0}\frac{||Ax||}{||x||},$$

be a norm (that is, satisfy the norm axioms) on $M_n(\mathbb{R})$. If we drop the sup and instead use a fixed $x$, then $\sigma$ doesn't satisfy the first axiom ($\sigma(A)=0\Leftrightarrow A=0)$ any more.

• @srijan thanks!
– jkn
Commented May 16, 2013 at 19:49
• nice explanation @jkn Commented May 17, 2013 at 4:07
• glad its of any use.
– jkn
Commented May 17, 2013 at 12:08