Interpolation, Extrapolation and Approximations rigorously

A foreign book mentioned that

"when the Lagrange's interpolation formula fails (for example with large sample due to Runge's phenomenon), you should use approximation methods such as Least-squares-method."

I am confused because I have always thought that interpolation/extrapolations are approximations. My confusion lies in the fact that the book used the three terms as disjoint while I have considered the first two terms as approximating. So how do you define the terms more rigorously?

For example, can the Lagrange polynomial (also known as Lagrange interpolation) be extrapolation, interpolation and approximation at the same time? I would say yes and cannot see no problem to use the L polynomial to create extrapolations and approximations (I feel the terms fuzzy).

-

"Approximation" is a very general term, whose definition I'll defer to a later paragraph. For the time being, I concentrate on interpolation and extrapolation.

The assumption here is that you are given a set of points $(x_k,y_k),\quad k=1\dots n$. Interpolation corresponds to finding a function $f(x)$ (e.g. a polynomial) such that $y_k=f(x_k)$ for all $k$.

Usually, the points $(x_k,y_k)$ are within some interval or region, e.g. $x_1 < x_2 < \dots < x_n$. Extrapolation is essentially evaluating the interpolating function $f(x)$ at a value of $x$ outside the region/interval containing the original points, e.g. some value $w < x_1$, or $z > x_n$. This can be slightly risky, since an interpolating function being well-behaved at a neighborhood of the interpolation points does not at all guarantee that the function will still behave nicely outside the original interval/region (and even then you see things like the Runge phenomenon, where the interpolant wiggles widely in between interpolation points).

Approximation, as I've said, is a very general term. One usually does not speak of interpolants as "approximants", since interpolants pass through each given point by construction. However, by a slight abuse of terminology, one sometimes considers the evaluation of some "difficult" function $f(x)$ by taking its values at known points, and constructing an interpolant from these known points. In this respect, the "interpolant" here is called an "approximant". (In my opinion, this (usually) isn't the best way to obtain an approximant!)

One might also consider the more general case of the given points having some form of error; here, one usually fits instead of interpolating. I talked about that in this answer, so I won't be repeating myself...

-
you used the term "error" to differentiate interpolation/extrapolation from approximation, in your old answer. Suppose you have random data points and you need to create interpolation. How do you know which metric to use and how do you call the error with metric? Is it random error, systematic error or something else? So are you sure you do not have error -term also with interpolation and extrapolation? Sorry I feel this difference fuzzy. – hhh Sep 18 '11 at 17:42
I said there: "Fitting on the other hand assumes your data is contaminated with error, and you want the polynomial that is the 'best approximation' to your data." :) "One treats 'exact data' and 'error-contaminated data' differently" would be a short version of my old answer. You of all people should know which best to choose, since you know where your data came from! – J. M. Sep 18 '11 at 17:46
...and now you are skipping my critique, sorry but ad hominem is futile. I see no reason to believe that some model is an exact presentation. Perhaps, "method of multiple scales" may be useful to try different interpolation/extrapolation techniques (even though they are "exact"), investigating.... – hhh Sep 18 '11 at 17:51
That was not in any event an ad hominem, and it is true that you should know where your data came from before choosing what to use. If they're exact values, interpolate; if they're error-contaminated, like measurements from the physical world, you fit. Again, know the source. – J. M. Sep 18 '11 at 17:55
P.S. A cursory glance at old books of the difference calculus would net you interpolation formulae with "error terms"; this was important in the days the only way to compute complicated functions like the sine or arctangent was to approximate them locally with an interpolating polynomial through chosen points... – J. M. Sep 18 '11 at 17:57

Interpolation or extrapolation produces an exact formula, not an approximation, for a polynomial that matches given data. However, this might be used as an approximation to the unknown function that produced those data.

-
yes but interpolation/extrapolation depends on the metric you are using, square metric returns totally different result to infinite metric for example. Interpolation per se is not just Lagrange Interpolation, there must be more into this. Now I cannot see how interpolation/extrapolation are not always approximations. There are many decisions such as metric which you can do wrong, hence approximation and not exact. – hhh Sep 18 '11 at 17:32