Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am taking a machine learning course and today we were given an example of regression, with two attributes $x_,x_2$ and $y$ being the real valued outcome.
$y$ is a quadratic function of $x_1,x_2$, given as
$$y=\alpha_1 x_1^2+\alpha_2 x_2^2+\alpha_3 x_1x_2+\alpha_4x_1+\alpha_5x_2+c$$ Now my teacher told that we can view this function as a linear function of $\lt x_1^2,x_2^2,x_1x_2,x_1,x_2 \gt$.
But I can't really visualize how can a curved function look like a straight line.
Please some one explain it or give some good reference.

share|improve this question
1  
It is a strange way to look at it. You can define the linear (actually affine) function $f(z_1,z_2,z_3,z_4,z_5) = c+\sum_{i=1}^5 \alpha_i z_i$, and then you have $y = f(x_1^2,x_2^2,x_1 x_2 , x_1, x_2)$, but its not clear why you want to do that. –  copper.hat Jul 27 '12 at 7:19
1  
Perhaps the best idea is not to think of it geometrically but algebraically. We need to pretend $x_1^2,x_2^2,x_1x_2,x_1,x_2$ are five independent variables, and I don't think attempting to visualize five-dimensional hyperplanes in relation to a quadratic surface in three dimensions is the way; just work formally. BTW, in what context are you supposed to view it as a linear function? The details of the specific application might lend to a more narrowly tailored explanation behind why such an interpretation works. –  anon Jul 27 '12 at 7:19
    
@anon It is related to support vector machines. Although he hadn't taught about SVMs but he just gave an idea that we can view every function as a straight line in higher dimensions. –  Happy Mittal Jul 27 '12 at 7:24
    
I haven't time for a complete answer, but Figures 93 and 94 on this page show how the simpler case of how $f(x,y) = ax + by + c(x^2+y^2)$ can be visualized as a linear function of $(x,y,x^2+y^2)$. The point is that a nonlinear function of the original data is turned into a linear function of a curved warping of the data. –  Rahul Jul 27 '12 at 7:52
    

1 Answer 1

It would seem to me that your teacher has completely botched his/her explanation. What s/he probably intended to say is that your model function ($y$ as a function of the parameters $\alpha_k$) is what's termed as a linear model (and thus, the kind of regression you need to do here is called a linear regression).

Here, you can think of the adjective "linear" like so: if you take the partial derivative of $y$ with respect to any of the $\alpha_j$, you will obtain an expression that is free of any of the $\alpha_j$ (in fact, the result will be whatever was multiplied with the $\alpha_j$ under consideration). As an example,

$$\frac{\partial y}{\partial \alpha_4}=x_1$$

See also this answer in CV.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.