Find Least Squares Regression Line

I have a problem where I need to find the least squares regression line. I have found $\beta_0$ and $\beta_1$ in the following equation

$$y = \beta_0 + \beta_1 \cdot x + \epsilon$$

So I have both the vectors $y$ and $x$.

I know that $\hat{y}$ the vector predictor of $y$ is $x \cdot \beta$ and that the residual vector is $\epsilon = y - \hat{y}$.

I know also that the least squares regression line looks something like this $$\hat{y} = a + b \cdot x$$ and that what I need to find is $a$ and $b$, but I don't know exactly how to do it. Currently I am using Matlab, and I need to do it in Matlab. Any idea how should I proceed, based on the fact that I am using Matlab?

Correct me if I did/said something wrong anyway.

• I am not at all familiar with Matlab. However, do you know the formulas for the least-squares estimates of $a$ and $b$? – Clarinetist Dec 2 '15 at 18:15
• @Clarinetist Are you talking about the formulas for $b_1$ and $b_2$ appearing here: ch.mathworks.com/help/curvefit/least-squares-fitting.html? – nbro Dec 2 '15 at 18:22

First define

X = [ones(size(x)) x];

then type

regress(y,X)

Observations:

• the first step is to include a constant in the regression (otherwise you would be imposing $a=0$).

• the output will be a vector with the OLS estimates $(a,b)$.

• You mean something like this: X = [ones(length(x), 1), x]; ? – nbro Dec 2 '15 at 18:25
• @nbro That will achieve the same goal. – mzp Dec 2 '15 at 18:27
• Yes, I understood that the output should be a vector (a, b) in this case. But, by using the formulas for $b_1$ and $b_2$ here I would then find the same results, right? – nbro Dec 2 '15 at 18:29
• @nbro Yes, that is correct. – mzp Dec 2 '15 at 18:31
• But in the case I would like to use those equations I would need to find $\beta$ first, i.e. the coefficients of the real line in order to defined $\hat{y}$ (as I did), right? – nbro Dec 2 '15 at 18:34

Sequence of $m$ measurements: $$\left\{ x_{k}, y_{k} \right\}_{k=1}^{m}$$ Model: $$y(x) = \beta_{0} + \beta_{1} x$$ Linear system: \begin{align} % \mathbf{A} \, \beta &= y \\ % A \left[ \begin{array}{cc} 1 & x_{1} \\ 1 & x_{2} \\ \vdots & \vdots \\ 1 & x_{m} \end{array} \right] % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= \left[ \begin{array}{c} y_{1} \\ y_{2} \\ \vdots \\ y_{m} \end{array} \right] % \end{align} Least squares solution: $$\beta_{LS} = \left\{ \beta \in \mathbb{C}^{2} \colon \lVert \mathbf{A} \,x - y \rVert_{2}^{2} \text{ is minimized} \right\}$$ Solution type: we have full column rank. Solution is unique - a point.

Solution method 1: Normal equations \begin{align} % \mathbf{A}^{*} \,\mathbf{A} \, \beta &= \mathbf{A}^{*} \,y \\ % A \left[ \begin{array}{cc} \mathbf{1} \cdot \mathbf{1} & \mathbf{1} \cdot x \\ x \cdot \mathbf{1} & x \cdot x \end{array} \right] % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= \left[ \begin{array}{c} \mathbf{1} \cdot y \\ x \cdot y \end{array} \right] % \end{align} $$\Downarrow$$ \begin{align} % \beta &= \left( \mathbf{A}^{*} \, \mathbf{A} \right)^{-1} \mathbf{A}^{*} \,y \\ % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= % inv \left( \left( \mathbf{1} \cdot \mathbf{1} \right) \left( x \cdot x \right) - \left( \mathbf{1} \cdot x \right)^{2} \right)^{-1} \left[ \begin{array}{rr} x \cdot x & -\mathbf{1} \cdot x \\ -\mathbf{1} \cdot x & \mathbf{1} \cdot \mathbf{1} \end{array} \right] % \left[ \begin{array}{c} \mathbf{1} \cdot y \\ x \cdot y \end{array} \right] % \end{align}

Solution method 2: Moore-Penrose pseudoinverse: $$\beta = \mathbf{A}^{+} y$$

The MATLAB intrinsic mldivide is one option.