# How do we change the basis of a function?

Consider vector space $\mathbb{F}$ all funcitions with domain and range $\mathbb{R}$

Now, consider the subset of functions, $$f_{k}(x)= 1; x=k$$ $$=0; x\neq k$$

One such function is defined for each real $k$, so there are an infinite number of these functions. Clearly, all these functions (one for each $k$) cannot be linearly combined to give a net $0$ result. That means an infinite number of elements of $\mathbb{F}$ can be linearly independent. So, the dimensionality of $\mathbb{F}$ is infinity.

Clearly, any other function $g(x)$ in $\mathbb{F}$ can be represented as a linear combination of the $f_{k}(x)$ functions, with the component of $g(x)$ in $f_k(x)$ being $g(k)$. So, am I right in saying that this subset of $f_{k}(x)$ functions is a basis for the vector space of functions, kind of like the $i, j, k$ unit vectors for arrows?

If yes, then, how do we find the other non-obvious basis for functions? I mean it was obvious that all $f_{k}(x)$ are linearly independent. If any infinite set of functions is given, how do we find out that they are linearly independent? And if they are linearly independent, then how do we calculate the components of any other function in $\mathbb{F}$ in that basis?

Clearly, all these functions (one for each $k$) cannot be linearly combined to give a net $0$ result. That means an infinite number of elements of $\mathbb{F}$ can be linearly independent.

Correct. So the dimension of $\mathbb{F}$ is indeed infinite, although as far as we know at the moment that still can be any kind of infinity (until we do further work to determine precisely this cardinality).

Clearly, any other function $g(x)$ in $\mathbb{F}$ can be represented as a linear combination of the $f_k(x)$ functions, …

No, that's not true. In the context of vector spaces without any additional structures, a linear combination means a finite linear combination. Taking infinite linear combinations requires taking some kind of a limit; in other words, it requires the notions of convergence, for which we need to have a topological vector space. Without any topology, like in this example, we can't speak of infinitely long sums.

So, am I right in saying that this subset of $f_k(x)$ functions is a basis for the vector space of functions?

For the reason explained above, no. According to the definitions of being linearly independent and of being a basis, these functions are linearly independent, but they do not form a basis — roughly speaking, because there are too few of them and they can't generate all the functions from $\mathbb{R}$ to $\mathbb{R}$ (elements of the space $\mathbb{F}$).

The really bad news is that the dimension of this vector space is uncountably infinite. So we can assert that it has a basis, but I'm afraid we can't realistically reveal or construct it. (Note that the assertion that any vector space has a basis is equivalent to the axiom of choice.)

• As you wrote it, it sounds as if for no vector space with uncountable dimension we could specify an explicit basis. Which is of course wrong; for example, the space of real functions with finite support is indeed spanned by the functions $f_k(x)$, which is an uncountable basis because $\mathbb R$ is uncountable. – celtschk Nov 20 '17 at 6:27
• But what about Taylor and Fourier series? Doesn't the existence of Taylor series mean that the set of functions {$1, x, x^2, x^3..........$} form a basis for all other functions from $\mathbb{R}$ to $\mathbb{R}$? – Ryder Rude Nov 20 '17 at 6:41
• @celtschk Now when I think about it, doesn't expressing a function $g(x)$as linear combination of $f_{k}(x)$ functions mean that we are adding an infinite number of 0's to $g(k)$? The sum of an infinite number of 0's is undefined. – Ryder Rude Nov 20 '17 at 6:45
• @RyderRude: In a finite linear combination, you are only adding finitely many zeros. Of course you form infinitely many such sums, but that is no problem, as each individual such sum has only finitely many terms. About the Taylor series: This only exists for infinitely often differentiable functions, which are already the minority of all function. And you don't even get all of those from power series (try the Taylor expansion of $\exp{-1/x^2}$ around $x=0$). Similar objections also apply for the Fourier series. – celtschk Nov 20 '17 at 7:19
• @celtschk: Agreed, it was poorly worded. Thank you for the correction! – zipirovich Nov 20 '17 at 15:51

Once we go to infinity, things get more complicated.

A basis is traditionally defined as a finite linear combination of elements (a Hamel basis). Countable linear combinations of elements are defined in terms of a Schrauder basis.

Think about some function $f:\mathbb{R}\to\mathbb{R}$. Say we know it's value on each real number $r\in\mathbb{R}$. Then, we can try to write it in terms of your basis: $$f(x) = \sum_{r\in\mathbb{R}}f_k(x)f(r)$$ Oops --- this is where we have an issue. You've probably seen in the past that: $$\sum_{n = 1}^\infty f(n) = \lim_{n\to\infty}\sum_{i = 1}^nf(i)$$ What do we do when the left sum is $\sum_{r\in\mathbb{R}}$? One thing we can't do is write the sum in some particular order. By this, I mean it can't be written as: $$f(r_1)+f(r_2)+\dots$$ As this implies that we can enumerate all the $r\in\mathbb{R}$, so $|\mathbb{R}| = |\mathbb{N}|$, which is false.

There are certain things we can do to try to fix this, but the general technique I've seen has been to stick with Schrauder/Hamel basis.

• You mean that all $f_{k}(x)$ are linearly independent but still they can't be used as a basis? – Ryder Rude Nov 20 '17 at 5:28
• I get it. $\sum_{r\in \mathbb{R}}f_{k}(x)f(r)$= f(r) + an infinite number of 0's which is not defined. So, what is this Hamel basis? Is it an infinite set of functions? – Ryder Rude Nov 20 '17 at 5:32
• I mean that linear dependence/independence depends on how big of sums you're allowed to take. It may be that (even with an infinite set of vectors) that there are no linear relations between a finite sum of them even if there are between an infinite sum of them. An example is the set of rational functions $\frac{f(x)}{g(x)}$ (both $f(x)$ and $g(x)$ polynomials). Then, $\{1,x,\dots,x^n,\dots\}\cup \{\frac{1}{1-x}\}$ is a linearly independent set if we only allow finite sums, but dependent if we allow infinite sums. – Mark Nov 20 '17 at 5:32
• Are the basis in Taylor series and Fourier transform Shrauder basis? If so, can a Shrauder basis be made from any infinite set of functions? Because I think I read that, in case of a finite n dimensional vector space, it is possible to make a linearly independent basis from any $n$ vectors. – Ryder Rude Nov 20 '17 at 5:46
• First of all, Schauder, not Schrauder. Second, the thing with Schauder bases is not just that they allow infinitely (but countably) many elements. Since an infinite sum requires taking limits, the concept of Schauder bases only makes sense in topological vector spaces (for example, Banach spaces, but not only). The vector space of all functions from $\mathbb{R}$ to $\mathbb{R}$... hmmm, I don't know of any reasonable topology there. – zipirovich Nov 20 '17 at 5:47