# What is the meaning of null regression coefficients

I have real data $x(t)$ which I model using the linear regression model: $\displaystyle f(t)= \sum_{i=0}^n c_i \phi_i(t)$, where $c_i$ are the regression coefficients I'm trying to find and $\phi_i(t)$ are a basis function (I chose Legendre Polynomials for their orthogonality).

I noticed something strange I couldn't explain i.e. when I fix $n=1$ for my model I get $\hat{c}_0, \hat{c}_1$, normal! Then when I fix $n=2$ for my model I get $\hat{c'}_0=\hat{c}_0, \hat{c'}_1=\hat{c}_1, \hat{c'}_2=0$ (Seems like $n=2$ didn't add anything to the data modeling!)

What is more strange is that when I fix $n=3$, I get $\hat{c''}_0, \hat{c''}_1, \hat{c''}_2, \hat{c''}_3$ that are completely different (no $\hat{c''}_2=0$)

Does anyone understand what is happening here?

This also happens for other even index coefficients $c_4, c_6, \dots$ i.e. when they are the latest regression coefficient in my model they are null but when I add another odd index coefficient they are not. Also I noticed that the coefficient values I obtain are high (as if to indicate an overfitting!)

• Can you list the coefficients so we see, for example, if they just look like a change of basis from Legendre polynomials to odd degree powers of x (or something like that)? – zyx Oct 23 '16 at 2:17
• here are the coeff. wrt $n$: ($n=1: \hat{c}_0=-47.85, \hat{c}_1=-29.08$), ($n=2: \hat{c'}_0=-47.85, \hat{c'}_1=-29.08, \hat{c'}_2 = 0.00$), ($n=3: \hat{c''}_0=-2.26\times 10^3, \hat{c''}_1=-2.44\times 10^3, \hat{c''}_2 = -1.14\times 10^3, \hat{c''}_3 = -0.22\times 10^3$) – Learn_and_Share Oct 23 '16 at 2:27
• thanks. can you write out what the polynomial approximation to f(t) is at each value of n (the $\sum c_i \phi_i (t)$), to see if it is converging to anything? – zyx Oct 23 '16 at 2:30
• my f(t) has more than 10000 points! I don't think I can do that! – Learn_and_Share Oct 23 '16 at 2:33
• What I noticed is that I can recover good enough the form of my real data using $n=3$ but not $n=2$ or $n=1$. – Learn_and_Share Oct 23 '16 at 2:34

## 1 Answer

The Legendre polynomials are orthogonal, but your regression results (by changing) show that their values on your data have nonzero covariance.

If $f(t)$ is odd that would explain the odd/even pattern with the $c_i$, where the even degree $c_{2n}$ contribute nothing on their own, but are involved in isolating the contribution of odd powers of $x$.

• what do you mean by nonzero covariance? Since I have different real data measurements for which I use this model, I noticed also that the regression coefficients I'm estimating are a lot correlated (they are linear combinations of each other)! Is this a consequence of this nonzero covariance? – Learn_and_Share Oct 23 '16 at 2:31
• Also what is it you mean by "isolating the contribution of odd powers of $x$"? – Learn_and_Share Oct 23 '16 at 2:38
• If f(t) is an odd function then adding the next higher even-degree polynomial to the regression will not change the regression output. However there is a unique linear combination of the c_i (of all degrees, even and odd, up to 2k+1, for a fixed value of k) that has a given set of coefficients for odd powers of x, and zero for all the even degree powers. – zyx Oct 23 '16 at 3:33
• Nonzero covariance = ($\phi_i(t)$ and $\phi_j(t)$ are correlated, on the set of $t$ used in the regression). – zyx Oct 23 '16 at 5:13
• can $phi_i(t)$ and $\phi_j(t)$ be correlated but orthogonal? – Learn_and_Share Oct 23 '16 at 5:24