I'm interested in understanding the usual inner product on functions spaces more deeply than I already do. That is, the inner product $\int f(t) \;g^*(t) \;dt$, where $f$ and $g$ are complex valued functions over whatever domain.
In my research so far, I've seen analogies drawn between this inner product, and how it is essentially like a dot product in $C^n$. For example, we have $(v_1, v_2, ..., v_n) \cdot (u_1, u_2, ..., u_n) = \sum_{i=1}^n v_i u_i ^ *$, and this is a bit like considering $f(x) g(x)^*$ for each $x$, and then integrating over the domain.
I find this (so far) unsatisfactory for a number of reasons. Firstly, it is not clear in what sense the image of each point in the domain represents a 'dimension' of the function space. Secondly, it isn't clear that what works in finite dimensions can naturally be extended to infinite dimensions. I understand that the above is only really meant to be an analogy rather than some proper argument, but I don't find analogies very helpful unless the situations actually are analogous.
Essentially, I would like to see a motivation for this inner product on function spaces. Why would somebody come up with it, if they had never seen it before? I know that $\sin(x)$ and $\cos(x)$ are orthogonal, but only because their inner product as above is zero. I'm convinced that, without recourse to this integral, there is still a sense in which $\sin(x)$ and $\cos(x)$ are orthogonal, which would have naturally led to the construction of this inner product. I'm interested in finding this out.
Would anyone be able to provide some insight towards what I've discussed here?