2
$\begingroup$

Suppose we want to fit the finite and discrete set of data $\left\{(x_1, f(x_1), (x_2, f(x_2), (x_3, f(x_3), \ldots, f(x_n)\right\}$ to the regression model $g(x) = \displaystyle\sum_{k=0}^m \alpha_k x^k$.

$f$ is assumed to be at least continuous.

One way is to consider the model as a system of equations written as a matrix equation

$$\begin{bmatrix} y_1\\ y_2\\ y_3 \\ \vdots \\ y_n \end{bmatrix}= \begin{bmatrix} 1 & x_1 & x_1^2 & \dots & x_1^m \\ 1 & x_2 & x_2^2 & \dots & x_2^m \\ 1 & x_3 & x_3^2 & \dots & x_3^m \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & x_n & x_n^2 & \dots & x_n^m \end{bmatrix} \begin{bmatrix} \alpha_0\\ \alpha_1\\ \alpha_2\\ \vdots \\ \alpha_m \end{bmatrix}$$

and find a solution $\displaystyle\min_x \left\|b - Ax\right\|_2$ where $b = Ax$ as above, which can be obtained by solving $A^T A x = A^T b$.

An alternative approach is to assume $f$ as an element $\mathcal C[x_1, x_n]$ and project it onto a basis for $\mathcal P_m = \left\{p : p(x) = \displaystyle\sum_{k=0}^m \beta_k x^k \text{ for some sequence } \left\{\beta_k\right\}_{k=1}^m\right\} \cap \mathcal C[x_1, x_n]$ with respect to the discrete scalar product $\langle f, g \rangle = \displaystyle\sum_k f(x_k) g(x_k)$.

Hence, $\displaystyle\min_{\hat f \in \mathcal P_m} \left\| \hat f - f\right\|_2 = \displaystyle\sum_{k=1}^m \frac{\langle f, p_k \rangle}{\langle p_k, p_k \rangle}$ for some orthogonal basis $\mathcal B = \{ p_1, p_2, p_3, \ldots, p_m\}$ for $\mathcal P_m$.

Are these methods yielding the same analytical answer? If not, what can be said about how exact these approximations are and their numerical stability?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.