Suppose we want to fit the finite and discrete set of data $\left\{(x_1, f(x_1), (x_2, f(x_2), (x_3, f(x_3), \ldots, f(x_n)\right\}$ to the regression model $g(x) = \displaystyle\sum_{k=0}^m \alpha_k x^k$.

$f$ is assumed to be at least continuous.

One way is to consider the model as a system of equations written as a matrix equation

$$\begin{bmatrix} y_1\\ y_2\\ y_3 \\ \vdots \\ y_n \end{bmatrix}= \begin{bmatrix} 1 & x_1 & x_1^2 & \dots & x_1^m \\ 1 & x_2 & x_2^2 & \dots & x_2^m \\ 1 & x_3 & x_3^2 & \dots & x_3^m \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & x_n & x_n^2 & \dots & x_n^m \end{bmatrix} \begin{bmatrix} \alpha_0\\ \alpha_1\\ \alpha_2\\ \vdots \\ \alpha_m \end{bmatrix}$$

and find a solution $\displaystyle\min_x \left\|b - Ax\right\|_2$ where $b = Ax$ as above, which can be obtained by solving $A^T A x = A^T b$.

An alternative approach is to assume $f$ as an element $\mathcal C[x_1, x_n]$ and project it onto a basis for $\mathcal P_m = \left\{p : p(x) = \displaystyle\sum_{k=0}^m \beta_k x^k \text{ for some sequence } \left\{\beta_k\right\}_{k=1}^m\right\} \cap \mathcal C[x_1, x_n]$ with respect to the discrete scalar product $\langle f, g \rangle = \displaystyle\sum_k f(x_k) g(x_k)$.

Hence, $\displaystyle\min_{\hat f \in \mathcal P_m} \left\| \hat f - f\right\|_2 = \displaystyle\sum_{k=1}^m \frac{\langle f, p_k \rangle}{\langle p_k, p_k \rangle}$ for some orthogonal basis $\mathcal B = \{ p_1, p_2, p_3, \ldots, p_m\}$ for $\mathcal P_m$.

Are these methods yielding the same analytical answer? If not, what can be said about how exact these approximations are and their numerical stability?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.