Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

This is with reference to projection pursuit regressions. I kind of get the idea behind approximating a continuous function using weighted sums of ridge functions. I am not sure why ridge functions work in the model. Or put another way, why work with a multivariate function that has values in one direction only? Why not just use a univariate function instead?

Also, the model works by generating 1 dimensional projections of multidimensional data. I understand the need for dimensionality reduction (curse of dimensionality) but why is looking for structure in the data's 1 dimensional projection equivalent to looking for structure in the data itself?

I hope I was able to frame the question right. Any help would be much appreciated

thanks a lot in advance !

share|improve this question

1 Answer 1

If you use a model where the dependent variable is a function of a single variable then you are doing standard nonparametric regression and you can use kernel regression, splines, local polynomials, etc. The problem is often that you don't know what variable you want to regress on or more than one variable is important.

You are right that the basic motivation for projection pursuit regression is the curse of dimensionality. Unfortunately, I think your question stems from a flawed premise. That is, there is no equivalence between finding structure in a projection onto a subspace and finding structure per se. Just because your data has structure does not mean projection pursuit regression will find it. If there was such a guarantee, then there would be no curse of dimensionality. The method may still provide good results for your problem though. The basic question of course is whether or not a relatively small number of directions captures most of whatever is interesting in the data. The nice thing about projection pursuit regression, compared to say, principle component regression, is that you can learn nonlinear relationships in these directions. In practice, it is somewhat rare that a practitioner would actually have an a priori reason to think projection pursuit regression will learn the true model. The directions and associated smooth functions are normally difficult to interpret and the results are accepted if they perform acceptably in out-of-sample prediction. It is hard to do much better for general high-dimensional problems.

I have a (non-comprehensive) list of suggestions for you. 1. Try doing PCA on the data first. If any of these directions seem sensible then you might be better off doing one-dimensional regressions on these. 2. Try generalized additive models, which do not project the data. If you do not think that linear combinations of the data are interesting then avoid regressing on them. 3. You may be able to throw away a lot of your variables without losing much. If you can pare down the number of variables then projection pursuit regression or generalized additive models may give better results. There are quite a few methods for doing this, e.g. sparse additive models, forward/backward stepwise selection, and using bandwidth selection techniques to "smooth out" uninteresting variables.

I hope this helps.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.