Understanding of the theorem that all norms are equivalent in finite dimensional vector spaces

Question

The following is a well-known result in functional analysis:

If the vector space $X$ is finite dimensional, all norms are equivalent.

Here is the standard proof in one textbook. First, pick a norm for $X$, say $$\|x\|_1=\sum_{i=1}^n|\alpha_i|$$ where $x=\sum_{i=1}^n\alpha_ix_i$, and $(x_i)_{i=1}^n$ is a basis for $X$. Then show that every norm for $X$ is equivalent to $\|\cdot\|_1$, i.e., $$c\|x\|\leq\|x\|_1\leq C\|x\|.$$ For the first inequality, one can easily get $c$ by triangle inequality for the norm. For the second inequality, instead of constructing $C$, the Bolzano-Weierstrass theorem is applied to construct a contradiction.

The strategies for proving these two inequalities are so different. Here is my question,

Can one prove this theorem without Bolzano-Weierstrass theorem?

UPDATE:

Is the converse of the theorem true? In other words, if all norms for a vector space $X$ are equivalent, then can one conclude that $X$ is of finite dimension?

Answer 1

To answer the question in the update:

If $(X,\|\cdot\|)$ is a normed space of infinite dimension, we can produce a non-continuous linear functional: Choose an algebraic basis $\{e_{i}\}_{i \in I}$ which we may assume to be normalized, i.e., $\|e_{i}\| = 1$ for all $i$. Every vector $x \in X$ has a unique representation $x = \sum_{i \in I} x_i \, e_i$ with only finitely many nonzero entries (by definition of a basis).

Now choose a countable subset $i_1,i_2, \ldots$ of $I$. Then $\phi(x) = \sum_{k=1}^{\infty} k \cdot x_{i_k}$ defines a linear functional on $x$. Note that $\phi$ is not continuous, as $\frac{1}{\sqrt{k}} e_{i_k} \to 0$ while $\phi(\frac{1}{\sqrt{k}}e_{i_k}) = \sqrt{k} \to \infty$.

There can't be a $C \gt 0$ such that the norm $\|x\|_{\phi} = \|x\| + |\phi(x)|$ satisfies $\|x\|_\phi \leq C \|x\|$ since otherwise $\|\frac{1}{\sqrt{k}}e_k\| \to 0$ would imply $|\phi(\frac{1}{\sqrt{k}}e_k)| \to 0$ contrary to the previous paragraph.

This shows that on an infinite-dimensional normed space there are always inequivalent norms. In other words, the converse you ask about is true.

Answer 2

You are going to need something of this nature. A Banach Space is a complete normed linear space (over $\mathbb{R}$ or $\mathbb{C}$). The equivalence of norms on a finite dimensional space eventually comes down to the facts that the unit ball of a Banach Space is compact if the space is finite-dimensional, and that continuous real-valued functions on compact sets achieve their sup and inf. It is the Bolzano Weirstrass theorem that gives the first property.

In fact, a Banach Space is finite dimensional if and only if its unit ball is compact. Things like this do go wrong for infinite-dimensional spaces. For example, let $\ell_1$ be the space of real sequences such that $\sum_{n=0}^{\infty} |a_n| < \infty $. Then $\ell_1$ is an infinite dimensional Banach Space with norm $\|(a_n) \| = \sum_{n=0}^{\infty} |a_n|.$ It also admits another norm $\|(a_n)\|' = \sqrt{ \sum_{n=0}^{\infty} |a_{n}|^2}$ , and this norm is not equivalent to the first one.

Answer 3

Theorem. Let $X$ be a real or complex finite-dimensional vector space, and let $\lVert\cdot\lVert_a$ and $\lVert\cdot\lVert_b$ be norms on $X$. Then $\lVert\cdot\lVert_a$ and $\lVert\cdot\lVert_b$ are equivalent, in the sense that there are constants $C,D>0$ such that for every $x\in X$, $$C\lVert x\lVert_b\leqslant \lVert x\lVert_a\leqslant D\lVert x\lVert_b\,.$$

I really like this proof by Steven G. Johnson at MIT. It uses Heine–Borel though, so it's not really any more elementary than the proof described in the original post that uses Bolzano–Weirstras; but perhaps it's at least a little bit cleaner/neater.

It consists of making the following four observations (it's a good exercise to try to prove them yourself, before looking at Johnson's much more detailed write-up):

1. Equivalence of norms is an equivalence relation. Hence, it's sufficient to show that every norm on $X$ is equivalent to $\lVert\cdot\rVert_1$, defined as in the original post.

2. It's sufficient to consider $x\in X$ with $\Vert x\Vert_1=1$, i.e. $x$ that lie on the unit sphere $S=\{x\in X:\Vert x\Vert_1=1\}$.

3. Every norm $\lVert\cdot\rVert$ on $X$ is continuous with respect to $\lVert\cdot\rVert_1$.

4. By Heine–Borel, the sphere $S$ is compact with respect to $\lVert\cdot\rVert_1$, and so, by the extreme value theorem, $\lVert\cdot\rVert$ must attain a minimum $m$ and a maximum $M$ on $S$.

Answer 4

Here's a proof of the equivalence of norms in finite dimension that does not use compactness arguments. It breaks down into three steps.

We set some notation first: given a normed finite dimensional real vector space $E$, write $E'$ for its algebraic dual space (set of all linear forms) and $E^{*}$ for its topological dual space (set of all continuous linear forms).

• All linear forms on $E$ are continuous, i.e $E'=E^{*}$, whatever the norm $\|.\|$ on $E$.

The Hahn-Banach theorem states that every continuous form on a subspace of $E$ can be extended to all of $E$. In particular, given $x\in E$, there is a $\varphi:E\rightarrow \mathbb{R}$ for which $\varphi(x)\neq 0$, since there are non trivial forms $\mathbb{R}x\rightarrow \mathbb{R}$ and these can be extended. As a consequence, the map \begin{align*} E &\rightarrow (E^{*})' \\ x &\mapsto (\phi\rightarrow \phi(x)) \end{align*} is injective. We also have isomorphisms $E'\simeq E$ and $E^{*}\simeq (E^{*})'$. Composing the three maps we have an injection $E' \rightarrow E^{*}$ so that $\dim E' \leqslant \dim E^{*}$. On the other hand since $E^{*}\subset E'$ we must have $\dim E^{*}=\dim E'$ and $E^{*}=E'$ as desired.

• All linear maps between finite dimensional normed spaces are continuous.

Let $A:F\rightarrow E$ be such a map and $e_{i},i\in I$ a basis for $E$. Let $(e_{i}^{*})$ be the dual basis of $(e_{i})$ and $\tilde{e_{i}}$ the linear map $\mathbb{R}\rightarrow E$ that maps $1$ to $e_{i}$. Then $Id_{E}=\sum_{i\in I}\tilde{e_{i}}e_{i}^{*}$. In particular, $A=\sum \tilde{e_{i}}e_{i}^{*}A$. The the $\tilde{e_{i}}$ are continuous and so are the $e_{i}^{*}A$, since they are linear forms. Hence $A$ is continuous as a sum of composites of continuous maps.

• All norms are equivalent.

Given two norms $\|.\|_{1}, \|.\|_{2}$ on $E$, the identity, being linear, defines an homeomorphism between $E,\|.\|_{1}$ and $E,\|.\|_{2}$. It is easy to check that this means that the norms are equivalent.

This proof is really a way of saying that the topology induced by a norm on a finite-dimensional vector space is the same as the topology defined by open half-spaces; in particular, all norms define the same topology and all norms are equivalent. There are other ways to prove that using the Hahn-Banach theorem.

Answer 5

One doesn't really need a different argument for each side of the inequality. If $\vert\vert \cdot \vert\vert_1,\vert\vert \cdot \vert\vert_2$ are two norms on a finite-dimensional vector space (over $\mathbb{R}$ or $\mathbb{C}$), then the restriction of $\vert\vert \cdot \vert\vert_1$ to the closed unit ball of $\vert\vert \cdot \vert\vert_2$ is a continuous function on a compact set (here the finite-dimensionality is used) and is therefore bounded from above by some $M > 0$. By positive homogeneity, it follows that $\vert\vert \cdot \vert\vert_1 \le M \vert\vert \cdot \vert\vert_2$. Switching the roles of $\vert\vert \cdot \vert\vert_1$ and $\vert\vert \cdot \vert\vert_2$, you get $\vert\vert \cdot \vert\vert_2 \le m \vert\vert \cdot \vert\vert_1$, hence $\frac{1}{m} \vert\vert \cdot \vert\vert_2 \le \vert\vert \cdot \vert\vert_1$, for some $m>0$.

Don't take this theorem too seriously though. This kind of equivalence relation between norms is pretty weak and two normed spaces with $\mathbb{R}^n$ as the underlying vector space can be completely different as far as their geometry is concerned (for instance, some norms come from an inner product [hence satisfy the nice geometric property which we call the "Parallelogram law"] and some don't).

Answer 6

Here's another answer to the question in the update. It's inspired by Geoff's answer and similar to tb's. tb's is superior in the sense that it shows how to construct unbounded linear functional on any infinite dimensional space. However, one might prefer the answer here because it just exhibits two very natural norms and shows they are inequivalent.

Let $X$ be an infinite dimensional space with basis $\{e_i\}_{i \in I}$. Each $x \in X$ has a unique representation of the form $x = \sum_{i \in I} c_i e_i$ with $c_i = 0$ for all but finitely many $i$. Define $\|x\|_1 = \sum_{i \in I} |c_i|$ and $\|x\|_{\infty} = \max_{i \in I} |c_i|$. It's easy to check these are indeed norms. For each $N \in \mathbb{N}$, there is an $x \in X$ such that $\|x\|_1 > N \| x \|_{\infty}$. Indeed, $x = \sum_{i=1}^{N+1} e_i$ has $\|x\|_1=N+1$ and $\|x\|_{\infty}=1$. Thus $\|{}\cdot{} \|_1$ and $\|{}\cdot{} \|_{\infty}$ are inequivalent.

Answer 7

Here is one approach, mentioned in Lang's analysis book.

Let $ V $ be an $ \mathbb{R} $-vector space with basis $ e_1 , \ldots, e_k $.

• $ \| x_1 e_1 + \ldots + x_k e_k \|_\infty := \max\{ |x_1|, \ldots, |x_k| \} $ is a norm on $ V $.

• $ \| \ldots \|_{\infty} $ makes $ V $ complete.

Let $ (v_j)_{j \geq 1} = (v_1, v_2, \ldots ) $ be a Cauchy sequence in $ V $. Exressing w.r.t basis vectors, $$ (v_1, v_2, \ldots ) = (e_1, e_2, \ldots, e_k) \begin{pmatrix} v_{11} &v_{12} &\ldots \\ v_{21} &v_{22} &\ldots \\ \vdots &\vdots &\ddots \\ v_{k1} &v_{k2} &\ldots \end{pmatrix}, \, \text{each }v_{ij} \in \mathbb{R} .$$ That is, $ v_{ij} $ is $ e_i $-coordinate of $ v_j $.
Notice sequence $ (v_{1j})_{j \geq 1} = (v_{11}, v_{12}, v_{13}, \ldots ) $ is Cauchy [because $ | v_{1n} - v_{1m} | \leq \| v_n - v_m \|_{\infty} $]. Similarly sequences $ (v_{1j}), (v_{2j}) , \ldots, (v_{kj}) $ are all Cauchy, hence convergent.
So let $ v_{1j} \to c_1, \ldots, v_{kj} \to c_k $ as $ j \to \infty $. Now $ v_j \to c_1 e_1 +\ldots + c_k e_k $ as $ j \to \infty $ [because $\| v_j - (c_1 e_1 + \ldots + c_k e_k) \|_\infty $ $ =\max \{ |v_{1j} - c_1| , \ldots, |v_{kj} - c_k | \} $ $ \to 0 $ as $ j \to \infty $], as needed.

• Now let $ \| \ldots \| $ be any norm on $ V $. We'll try to show $ \| \ldots \| $ and $ \| \ldots \|_{\infty} $ are equivalent on $ V $, by induction on dimension $ k $ of the normed space. So lets assume $ \| \ldots \| $ and $ \| \ldots \|_{\infty} $ are equivalent on every proper subspace of $ V $.

• Firstly, $$ \begin{align} \| x \| &= \| x_1 e_1 + \ldots + x_k e_k \| \\ &\leq |x_1| \| e_1 \| + \ldots + |x_k| \| e_k \| \\ &\leq \max \{|x_1 |, \ldots, |x_k|\} \left( \| e_1 \| + \ldots + \| e_k \| \right) \\ &= \| x \|_\infty C \end{align}$$ with $ C > 0 $.
  So we need only show $ \| x \|_\infty \leq D \| x \| $ for some $ D > 0 $.

---

Say there is no such $ D $. Then for every integer $ n > 0 $, there is a $ v_n $ with $ \| v_n \|_\infty > n \| v_n \| $.

Notice $ v_n \neq 0 $, i.e. each $ v_n $ has some non-zero coordinate.

So on defining $ w_n $ to be $ v_n $ divided by the coordinate of $ v_n $ with maximum absolute value, we have $ \| w_n \|_\infty = 1 $ and $ \| w_n \| = \| v_n \|_\infty ^{-1} \| v_n \| $ $\big($and hence $ \| w_n \| < \frac{1}{n} \big) $. Also each $ w_n $ has $ 1 $ as a coordinate.

Expressing sequence $ (w_n)_{n \geq 1} $ w.r.t basis vectors, $$ (w_1, w_2, \ldots ) = (e_1, e_2, \ldots, e_k) \begin{pmatrix} w_{11} &w_{12} &\ldots \\ w_{21} &w_{22} &\ldots \\ \vdots &\vdots &\ddots \\ w_{k1} &w_{k2} &\ldots \end{pmatrix}. $$ Every column has a $ 1 $, and every entry is $ \leq 1 $ in magnitude.

So there is a $ T \in \{ 1, 2, \ldots, k \} $ such that row $ w_{T 1}, w_{T 2}, \ldots $ has infinitely many $ 1 $s. Let $ J := \{ j : w_{T j} = 1 \} $ be the positions at which the $1$s occur in this row.

We'll now focus on sequence $ (w_j - e_T)_{j \in J} $.

Firstly, as $ (w_n)_{n \geq 1} $ itself converges to $ 0 $ w.r.t $ \| \ldots \| $, this sequence $ (w_j - e_T)_{j \in J} $ converges to $ (-e_T) $ w.r.t $ \| \ldots \| $.

Note the sequence $ (w_j - e_T)_{j \in J} $ lies in the subspace $ V_T := \{ x_1 e_1 + \ldots + x_k e_k : x_T = 0 \} $. It is also Cauchy in $ V_T $ w.r.t. $ \| \ldots \| $, because $ \| (w_n - e_T)-(w_m - e_T) \| = \| w_n - w_m\| $ $\leq \| w_n \| + \| w_m \| < \frac{1}{n} + \frac{1}{m}$.

But $ V_T $ is complete w.r.t $ \| \ldots \|_{\infty} $, and $ \| \ldots \| $ is equivalent to $ \| \ldots \|_{\infty} $ on $ V_T $. So $ V_T $ is complete w.r.t $ \| \ldots \| $. Hence $ (w_j - e_T)_{j \in J} $ must converge to a point in $ V_T $ w.r.t $ \| \ldots \| $, that is $ (-e_T) \in V_T $, a contradiction.

So there indeed is a $ D > 0 $ such that $ \| x \|_\infty \leq D \| x \| $ in $ V $, as needed.

Answer 8

Clearly, every norm in a one-dimensional vector space is equivalent, since $\|ae\|=|a|\thinspace \|e\|$ where $e$ is your basis vector, and so any two norms are in fact multiples of each other.

You can show easily that all one-dimensional vector spaces over complete fields (e.g., $\mathbf{R}$ and $\mathbf{C}$) are complete.

Now, suppose you've shown that all $n-1$-dimensional vector spaces are complete. Let $V$ be an $n$-dimensional vector space and let $\{e_1,\dots,e_n\}$ be a basis. Then $Span(e_1,\dots,e_{n-1})$ is a $n-1$-dimensional vector space, so it is complete. So it is closed in $V$. Let $r$ satisfy $B(e_n,r)\subset V\setminus Span(e_1,\dots,e_{n-1})$. If $a_n>0$, then $\|e_n-\sum_{k=1}^{n-1}(-a_k/a_n)\,e_k\|\geq r$, so $\|\sum_{k=1}^n a_k\,e_k\|\geq r|a_n|$.

You can do this with any of the basis vectors by renumbering, so $\sum_{k=1}^n |a_k|\leq C\|\sum_{k=1}^n a_k\,e_k\|$. Once you have that, you can show that your $n$-dimensional vector space is complete. Thus by induction, you have that all norms are equivalent to the 1-norm and also that all finite dimensional vector spaces are complete.