Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am stuck to this code and particularly the point F=[ones(size(x)) x x.^2]. What does it do mathematically? I cannot understand what the heck the matrix has to do with this least-squares regression.

I think it should do something like:

$$\begin{align} \min \sum_{i}&r_{i}\\ s.t. \\ r_{i}=(o_{i}- &h_{i})^2 \end{align} $$

where $r_{i}$ is the distance between the observed value $o_{i}$ and the plotting value $h_{i}$.

I do not have access to MATLAB so please refer to some free software such as R or Python if you are going to demo.

x=[1 3 5 7 9 11]'
y=[0.0100 2.3984 11.0256 4.0272 0.2408 0.0200]'
plot(x,y,'r*')
hold on                                 %  Model a_1+a_2*x+a_3*x^2
F=[ones(size(x)) x x.^2]
a=F\y                                   % solves F*a=y with Least-Squares
l=max(x)-min(x);                        % greatest distance between x -points
t=[floor(min(x)):l/100:ceil(max(x))]';  % plot distance a bit greater
plot(t,[ones(size(t)) t t.^2]*a,'b')    % [ones(size(t)) t t.^2]*a calcs fits in t
legend('Data points','Fits')
axis([0,12,-2,15])
SSE=sum((y-F*a).^2)                     % a sum
SST=sum((y-mean(y)).^2)                 % total sum
rSquared=1-SSE/SST                      % R-squared
fun = @(x)(-a(1)-a(2)*x-a(3)*x^2)       % -1 * fit, cos fminsearch looks for min
x_max=fminsearch(fun,6)
plot(x_max,a(1)+a(2)*x_max+a(3)*x_max^2,'o')
hold off

...this code is supposed to do the same thing:

x = [1 3 5 7 9 11]’;
y = [0.0100 2.3984 11.0256 4.0272 0.2408 0.0200]’;
X = [x.^0 x x.^2];               % Model: y = b1 + b2x + b3x^2 -> y = Xb
x_grid = 0:0.01:11;              % grid for drawing
b1_hat = inv(X’*X)*X’*y;         % Least squares
y1_hat = b1_hat(1)*x_grid.^0 + b1_hat(2)*x_grid.^1 +...
b1_hat(3)*x_grid.^2;
figure(1)                        % Drawing the fitted polynomial
plot(x,y,’*’,x_grid,y_hat)
optimal1 = max(y_hat)

...I am now confused by this X = [x.^0 x x.^2];, the logic seems the same as earlier but I cannot understand yet it.

share|improve this question
add comment

1 Answer

up vote 0 down vote accepted

The reason why the rows of your design matrix take the form $(1\quad x_k\quad x_k^2)$ is that you're fitting with the quadratic model

$$y=c_0+c_1x+c_2x^2$$

If you expand out the matrix product

$$\begin{pmatrix}1&x_1&x_1^2\\1&x_2&x_2^2\\\vdots&\vdots&\vdots\\1&x_m&x_m^2\end{pmatrix}\begin{pmatrix}c_0\\c_1\\c_2\end{pmatrix}$$

you should be able to recognize that in the residual function you're trying to minimize.

In general, one always adds a column of $1$'s to the design matrix to account for constant terms in the model.

share|improve this answer
    
b1_hat, y1_hat and b1_hat(3)... why and what it really do? It does vector multiplication with the X*X' (where X' is apparently transpose(?!)) and then it takes inverse but why, sorry I cannot understand this yet... –  hhh Dec 3 '11 at 8:34
1  
Maybe this answer on CV might clarify a few things to you. –  J. M. Dec 3 '11 at 8:44
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.