Let's say that I have a function $y = f_1(x)$. I have two other functions, $y = f_2(x)$ and $y = f_3(x)$, that I want to compare with the first function. Specifically, I want to see which one of $f_2$ and $f_3$ can best $approximate$ the function $f_1$. How do I go about doing this? All three functions are continuous.

I tried to think of an approach based on "error" that we see in regression analysis. For discrete data, we can calculate all kinds of errors that evaluate goodness of fits. For instance, if $f_1$ were a discrete function with values $\{y_1, y_2, \cdots , y_N\}$ corresponding to $\{x_1, x_2, \cdots, x_N\}$, the Mean Square Error (MSE) of $f_2$ would be given by $$ \text{MSE}_2 = \frac{1}{N} \sum_{i = 1}^N |y_i - f_2(x_i)|^2$$ and ditto for $f_1$. My question is, can we extend this to the case where $f_1$ is continuous? My initial thought was yes, since we can simply generalize the sum to an integral, and consider infinite data-points, meaning $N\to \infty$. This way, defining $\delta x \equiv 1/N$, I can argue $\delta x \to dx$ as $N \to \infty$. The new MSE would then be $$\text{MSE}_2 = \int_{a}^b |f_1(x) - f_2(x)|^2 \, dx.$$ I can then do the same for $f_3$ and compare the two results: whichever function gives me a lower MSE is a better approximate to $f_1$. Is this sound? Moreover, is this a good way to find out how well one function approximates the other, as is my initial goal?

  • $\begingroup$ For a sound answer, you should explain what is the purpose of doing this comparison. $\endgroup$ – user65203 Jul 27 '20 at 19:34

There are as many ways to measure how close two continuous functions (defined, say, on a closed and bounded interval $[a,b]$), as there are metrics on the set of those continuous functions.

The (square root of the) MSE$_2$ you are using is one metric on continuous functions. Another is the uniform metric, given by

$$d(f_1,f_2)=\max_{a\leq x\leq b}|f_1(x)-f_2(x)|$$

It is clear that MSE$_2$ is at most $(b-a)d(f_1,f_2)^2$, but two continuous functions can be close in the MSE$_2$ sense, but not so close in the uniform sense.

  • $\begingroup$ I see, so the "errors" measured by the two metrics can be different. Is there a proper name for my (unoriginally named) $\text{MSE}_2$ for continuous functions, or are they simply called different metrics? Could you also mention other metrics like these that I can consider? Thanks for the answer, by the way. $\endgroup$ – Yejus Jul 27 '20 at 19:33
  • $\begingroup$ @Yejus the mean square error is analogous to the Euclidean distance between points. If you minimize/maximize the square of a length you also minimize the length, so you skip the square root in the calculation. $\endgroup$ – CyclotomicField Jul 27 '20 at 19:35
  • 2
    $\begingroup$ Mean Square Error is a standard term. Mathematicians also call (the square root thereof) "Hilbertian norm", because it is the norm of a Hilbert space, denoted by $L^2(a,b)$. Other norms (hence, metrics) can be obtained by changing the power $2$ to $1\leq p<\infty$, and raising the whole integral to the power $1/p$. $\endgroup$ – uniquesolution Jul 27 '20 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.