Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I put this question here as it has a pure maths element to it, even though it has a statistical twist. Basically, I have been given the table of data:

$$\begin{matrix} i & \mathrm{Response} \, y_i & \mathrm{Covariate} \, x_i \\ 1 & -1 & -0.5 \\ 2 & -0.25 &-0.25 \\ 3 & 1 & 0 \\ 4 & 1.2 & 0.25 \\ 5 & 2.6 & 0.5 \end{matrix}$$

And for here, I need to create an orthogonal regression model. In other words, I need to come up with a model in the form

$$Y_i = \alpha_0 + \alpha_1(ax_i + b) + \alpha_2(dx_i^2 + ex_i + f)$$

Where

$$\sum_i^n ax_i + b = dx_i^2 + ex_i + f = 0$$

and $\alpha_{0,1,2}$ are just constants.

From looking at it, I can see that the linear term can just be $x_i$ as $\sum_i^n x_i = 0$, but I'm stuck on how to solve for the quadratic bit.

How can I do this?

share|improve this question
add comment

1 Answer

It looks as if you're using the (discrete) inner product

$$\langle f,g\rangle=\sum_j^n f(x_j) g(x_j)$$

and I'll proceed with that assumption.

The canonical method for producing a basis of orthogonal polynomials with respect to some (discrete or continuous) inner product is the Stieltjes procedure. I discussed the continuous case here; the discrete case proceeds similarly.

In any event, to summarize: if you have a set of monic orthogonal polynomials $\phi_k(x)$ satisfying the recurrence

$$\phi_{k+1}(x)=(x+b_k)\phi_k(x)+c_k\phi_{k-1}(x)$$

with the associated orthogonality condition $\langle \phi_j,\phi_k\rangle=0,\quad j\neq k$ and the initial values $\phi_{-1}(x)=0$ and $\phi_0(x)=1$, the recurrence coefficients $b_k$ and $c_k$ are obtained through the formulae

$$\begin{align*}b_k&=-\frac{\langle x\phi_k ,\phi_k\rangle}{\langle \phi_k,\phi_k\rangle}\\c_k&=-\frac{\langle\phi_k,\phi_k\rangle}{\langle\phi_{k-1},\phi_{k-1}\rangle}\end{align*}$$

where $x\phi_k$ denotes the product of $x$ and $\phi_k(x)$.

For this particular case, as you say, $\phi_1(x)=x$ is easily determined; thus, to compute $\phi_2(x)$, we compute $b_1=0$ and $c_1=-\frac18$ to yield $\phi_2(x)=x^2-\frac18$.

We can then determine that the quadratic least-squares fit to the data is (approximately) given by $0.71+3.46\phi_1(x)+0.285714\phi_2(x)$.

If you want more details on data-fitting with orthogonal polynomials, see this article by George Forsythe. You might also want to look up the Gram polynomials, which are the standard orthogonal polynomials used for data fitting when the data has equispaced abscissas.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.