Confused about the terminology of least regression squares I am confused by the language of my math text book where it says $\hat y = \alpha \times$basis function summation from $i=1$ to $n$? What are basis functions? And why does it says there are more data points than basis functions? I know how to derive $\alpha$ using linear algebra and calculus by using $mx + c$ as the equation of the regression line. 
Also I am a bit confused on the fact that how does it know that the smaller distance of the actual data point and $\hat y$ is perpendicular to the line of $\hat y$? I will be very very grateful if anyone can help me with this.
 A: A model which is linear with respect to the parameters write
$$y=\sum_{k=1}^n a_k\,f_k(x)$$ and the $f_k(x)$ are the $k$  basis functions.
Suppose that you want to fit data to the model
$$y=a_0+a_1\sin(x)+a_2\log(x)+a_3 e^{-\pi x}$$ Define $t_i=\sin(x_i)$, $u=\log(x_i)$, $v=e^{-\pi x_i}$. So the model is just
$$y=a_0+a_1t+a_2u+a_3v$$ which corresponds to a multilinear regression.
A: You are approximating the data points with a linear combination of independent functions. In the case of the straight line of equation
$$y=mx+p$$
there are two functions which are the identity,
$$y=x$$ and the unit,
$$y=1.$$
You can generalize with more functions, such as the powers $x^2,x^3,\cdots$ or any other. Basis function comes from the terminology of linear algebra.
Every data point lets you write an equation, such as
$$(x_k,y_k)\to y_k=mx_k+p.$$
When there are more points than basis functions, i.e. than unknown coefficients, the system is overdeterminate and has no solution. In this case, you try to find a good compromise, i.e. one that approximately satisfies all equations.

Notice that the minimization via the model $y=mx+p$ does not achieve the minimum distance to the lines, but the vertical distance to the line. See total least squares for the former. https://en.wikipedia.org/wiki/Total_least_squares
