Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.
x<-c(1,2,3)
y<-x^1.1+x
summary(lm(y~x+I(x^1.1)))

I have this code in R but it just is for the sake of easier understanding of what I am trying to achieve. If you execute the code, it will show the correct estimates, but it won't show p-values (shows NA) for each estimate, thus I am unable to test the significance. How can this be explained in a mathematical way?

P.S. if I add one more value to each of the vectors, p-value appears again. However, if I add another variable, let's say

summary(lm(y~x+I(x^1.1)+I(x^1.2)))

again p-values are all NAs

share|improve this question

1 Answer 1

up vote 1 down vote accepted

As you are finding out, when the number of your observations (n say) equals the number of parameters to be estimated (p say), you do not get any distributional results. I am sure (but not certain) that the many theorems for the linear model state somewhere that you need n>p. However the problem when $n=p$ can be seen more clearly by considering the t-statistics for testing the null hypothesis that $\beta_{0}$ say is zero (this could be the intercept for example). I believe (again I am not certain) that the t-statistics divides the estimate $\hat{\beta}_{0}$ by a quantity that is a function of the sum of squares residuals $\sum\nolimits_{i=1}^{n}(y_{i}-\hat{y}_{i})$ which in your case is zero since you have a perfect fit. This makes the denominator zero and so you cannot divide. Thus you cannot compute the test statistic. I think you will always get a perfect fit if $n=p$. In general in statistics there needs to be much more data than parameters for things to behave well.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.