# What does orthogonality mean in function space?

The functions $x$ and $x^2 - {1\over2}$ are orthogonal with respect to their inner product on the interval [0, 1]. However, when you graph the two functions, they do not look orthogonal at all. So what does it truly mean for two functions to be orthogonal? • You may want to take a look at this question. Also, the functions you plotted don't look "orthogonal" since you're not in function space. Think orthogonality=linear independence. Orthogonal vectors cannot be expressed one in term of the other (that's why we use them as bases). Similarly, orthogonal functions are linearly independent. In this case, no affine combination of $x$ yields $x^2-\frac{1}{2}$ and vice-versa. – Demosthene Mar 5 '15 at 18:04
• You should try graphing their product as well if you want to see a picture which might give you an intuition about orthogonality. If you work up the degrees you can find a third degree polynomial orthogonal to these two and then a fourth and fifth degree polynomial - you might also want to graph those to see what happens. You could also plot their products in pairs. This is a comment because it doesn't really answer the question "why not" but if you do some trying out you will get a better sense of what might be happening. – Mark Bennet Mar 5 '15 at 18:05
• Very loosely speaking, you can think of $f$ and $g$ as vectors whose components are their values at each point. Then if you multiply them componentwise and "add", i.e. integrate, you get this analogue of the Euclidean inner product. Of course this all has to be made precise! – Neal Mar 5 '15 at 20:13
• @Demosthene actually, orthogonality is a much stronger condition than linear independence. "Almost all" functions are linear independent with a given one, but only a small subset of those is actually orthogonal. (And there is the zero function, which is orthogonal but not linear independent.) – Paŭlo Ebermann Mar 5 '15 at 21:50
• @PaŭloEbermann You're absolutely right, of course. I probably should have been clearer in what I meant. Still, one cannot "see" orthogonality from plotting the functions - they are orthogonal in function space; and orthogonal functions can be used to form bases. Also, Mark Bennet's comment is more interesting than mine in that he provides a way of visualizing this orthogonality. – Demosthene Mar 5 '15 at 23:30

Consider these two functions defined on a grid of $x\in\{1,2,3\}$:

$$f_1(x)=\sin\left(\frac{\pi x}2\right),$$ $$f_2(x)=\cos\left(\frac{\pi x}2\right).$$

Their plot looks like If you look at their graph, they don't look orthogonal at all, as the functions plotted in the OP. Yet, being interpreted as vectors $(1,0,-1)^T$ and $(0,-1,0)^T$, they are indeed orthogonal with respect to the usual dot product. And this is exactly what is meant by "orthogonal functions" — orthogonality with respect to some inner product, not orthogonality of the curves $y=f_i(x)$.

Function spaces that have an inner product and are complete under the induced norm, i.e. Hilbert spaces, have their own sort of infinite-dimensional geometry. One should consider "functions" that belong to such a function space as "points" lying in an infinite-dimensional geometric space. In such spaces, the notion of "orthogonal functions" is interpreted geometrically, analogous to how in finite-dimensional Euclidean space we have a geometric notion of "orthogonal vectors."

Orthogonal vectors in a Hilbert space are, just like in Euclidean space, sort of the "most" linearly independent you can get. In Euclidean space, if $a$ and $b$ are vectors, then one can take the projection of $a$ onto $b$, say $p(a,b)$, and then one can write $a = p(a,b) + o(a,b)$, where $o(a,b)$ is the part of $a$ orthogonal to $b$. When $a$ and $b$ are orthogonal, $p(a,b)$ vanishes and you just have the orthogonal part $o(a,b)$, so in this sense no part of $a$ belongs to the subspace generated by $b$.

The same exact thing happens in infinite-dimensional Hilbert spaces (note my argument above used nothing about finite dimensions). We can't really draw infinite-dimensional space, so this geometric notion of orthogonality is interpreted abstractly.

• Can a function f, g be orthogonal over the interval (-inf, inf) if f ≠ -g? – Chinedum Ukejianya Mar 5 '15 at 21:20
• @ChinedumUkejianya I suppose a trivial example would be if one function were 0. A less trivial example might be $f(x) = x$ and $g(x) = \frac{1}{1+x^4}$ – Kyle Mar 6 '15 at 0:47
• @ChinedumUkejianya Yes. You can in fact create an orthonormal basis of such elements in any Hilbert space. See en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process to understand how this is done. For an explicit example, take any two functions that are orthogonal over $[0,1]$ and extend them to $(-\infty,\infty)$ by setting them to $0$ outside $[0,1]$. – Gyu Eun Lee Mar 6 '15 at 1:16

Since the inner product is $\left<f,g\right>=\int_{0}^{1}f(x)g(x)\mathrm{d}x$, then orthogonality means that the integral over the interval $[0,1]$ is zero.

• Yeah. But if the inner product is zero doesnt that mean that the functions need to be perpendicular to each other within [0 1], since f, g ≠ 0. – Chinedum Ukejianya Mar 5 '15 at 17:55
• Our intuition of "perpendicular" changes drastically once we are outside of the normal Euclidean inner product. – rnrstopstraffic Mar 5 '15 at 17:57
• Why does everyone upvote this answer? It completely misses the point of the question. The questioner asks for an intuition on what orthogonality really means... – Eff Mar 5 '15 at 17:59
• That is one interpretation; of course, the questioner asked what it truly means to be orthogonal. I answered the question that was asked. – rnrstopstraffic Mar 5 '15 at 18:06

The way in which the functions are orthogonal has only a very abstract resemblance to the way in which vectors in $\Bbb R^2$ are orthogonal. It does not mean that the graphs are geometrically orthogonal in any way; functions are not the same as their graphs. You should think of "orthogonal" as a metaphor: the functions are orthogonal because their inner product is zero. This is analogous to, but not the same as, the way two vectors in $\Bbb R^2$ point in geometrically perpendicular directions if their inner product (a different inner product) is zero.

Mathematics works by identifying common patterns and understanding them in a way that is at once more general and more abstract. By generalizing and abstracting the notion of the orthogonality of two vectors, we can apply the techniques of linear algebra to function spaces.

Functions can be added together, scaled by constants, and taken in linear combination--just like the traditional Euclidean vectors.

$$\vec{u} = a_1\vec{v}_1 + a_2\vec{v}_2 + \cdots a_n\vec{v}_n$$

$$g(x) = a_1f_1(x) + a_2f_2(x) + \cdots + a_nf_n(x)$$

Just as $\left( \begin{array}{c} 5\\ 2\\ \end{array} \right) = 5\left( \begin{array}{c} 1\\ 0\\ \end{array} \right) + 2\left( \begin{array}{c} 0\\ 1\\ \end{array} \right)$, I can say $5x + 2x^2 - 1 = 5(x) + 2(x^2 - \frac{1}{2})$

"Orthogonality" is a measure of how much two vectors have in common. In an orthogonal basis, the vectors have nothing in common. If this is the case, I can get a given vector's components in this basis easily because the inner product with one basis vector makes all other basis vectors in the linear combination go to zero.

If I know $\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right)$, $\left( \begin{array}{c} 0\\ 1\\ 0\\ \end{array} \right)$ and $\left( \begin{array}{c} 2\\ 0\\ -4\\ \end{array} \right)$ are orthogonal. I can quickly get the components of any vector in that basis:

$$\left( \begin{array}{c} 8\\ -2\\ 5\\\end{array} \right) = a\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) + b\left( \begin{array}{c} 0\\ 1\\ 0\\ \end{array} \right) + c\left( \begin{array}{c} 2\\ 0\\ -4\\ \end{array} \right)$$

$$\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) \cdot \left( \begin{array}{c} 8\\ -2\\ 5\\\end{array} \right) = \left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) \cdot \big[ a\left( \begin{array}{c} 1\\ 0\\ 0.5\\ \end{array} \right) + b\left( \begin{array}{c} 0\\ 1\\ 0\\ \end{array} \right) + c\left( \begin{array}{c} 2\\ 0\\ -4\\ \end{array} \right) \big]$$

I know the $b$ and $c$ term disappear due to orthogonality, so I set them to zero and forget all about them.

$$8.25 = 1.25 a$$

I can also get function basis components easily this way. Take the fourier series on $[0,T]$ for example, which is just a (infinitely long) linear combination of vectors/functions:

$$f(x) = a_0 + \sum_n^\infty a_ncos(\frac{2\pi n}{T}x) + \sum_n^\infty b_nsin(\frac{2\pi n}{T}x)$$

I know that all the $cos$ and $sin$ basis functions are orthogonal, so I can take the inner product with $cos(\frac{2\pi 5}{T}x)$ and easily get a formula for the $a_5$ coefficient because all the other terms vanish when I do an inner product.

$$\int cos(\frac{2\pi 5}{T}x) f(x) dx = a_5 \int cos^2(\frac{2\pi 5}{T}x) dx$$

Of course I could do this in general with $cos(\frac{2\pi q}{T}x)$ to get any of the $a_q$ components.

• for vectors, u=5v1+2v2, v1 and v2 cannot become a number, but, for function, g(5)=5*5+2(25-0.5)=74, 25 and 49 are orthogonal?? I am confused – fairytale Feb 16 '18 at 15:07
• @fairytale You shouldn't be focusing on the functions outputs like g(5), just as you shouldn't be focusing on the individual numbers inside a column vector. When talking about orthogonality, you need to look at the functions as a whole, just as you look at the column vectors as a whole. – eigenchris Feb 16 '18 at 19:34