# Find Least Squares Regression Line

I have a problem where I need to find the least squares regression line. I have found $\beta_0$ and $\beta_1$ in the following equation

$$y = \beta_0 + \beta_1 \cdot x + \epsilon$$

So I have both the vectors $y$ and $x$.

I know that $\hat{y}$ the vector predictor of $y$ is $x \cdot \beta$ and that the residual vector is $\epsilon = y - \hat{y}$.

I know also that the least squares regression line looks something like this $$\hat{y} = a + b \cdot x$$ and that what I need to find is $a$ and $b$, but I don't know exactly how to do it. Currently I am using Matlab, and I need to do it in Matlab. Any idea how should I proceed, based on the fact that I am using Matlab?

Correct me if I did/said something wrong anyway.

• I am not at all familiar with Matlab. However, do you know the formulas for the least-squares estimates of $a$ and $b$? Dec 2, 2015 at 18:15

First define

X = [ones(size(x)) x];


then type

regress(y,X)


Observations:

• the first step is to include a constant in the regression (otherwise you would be imposing $a=0$).

• the output will be a vector with the OLS estimates $(a,b)$.

• @nbro That will achieve the same goal.
– mzp
Dec 2, 2015 at 18:27
• @nbro Yes, that is correct.
– mzp
Dec 2, 2015 at 18:31
• @nbro Yes, you would.
– mzp
Dec 2, 2015 at 18:35
• @nbro Yes, that should work.
– mzp
Dec 2, 2015 at 18:45
• @nbro As far as I can tell this should work...
– mzp
Dec 2, 2015 at 19:04

Sequence of $m$ measurements: $$\left\{ x_{k}, y_{k} \right\}_{k=1}^{m}$$ Model: $$y(x) = \beta_{0} + \beta_{1} x$$ Linear system: \begin{align} % \mathbf{A} \, \beta &= y \\ % A \left[ \begin{array}{cc} 1 & x_{1} \\ 1 & x_{2} \\ \vdots & \vdots \\ 1 & x_{m} \end{array} \right] % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= \left[ \begin{array}{c} y_{1} \\ y_{2} \\ \vdots \\ y_{m} \end{array} \right] % \end{align} Least squares solution: $$\beta_{LS} = \left\{ \beta \in \mathbb{C}^{2} \colon \lVert \mathbf{A} \,x - y \rVert_{2}^{2} \text{ is minimized} \right\}$$ Solution type: we have full column rank. Solution is unique - a point.

Solution method 1: Normal equations \begin{align} % \mathbf{A}^{*} \,\mathbf{A} \, \beta &= \mathbf{A}^{*} \,y \\ % A \left[ \begin{array}{cc} \mathbf{1} \cdot \mathbf{1} & \mathbf{1} \cdot x \\ x \cdot \mathbf{1} & x \cdot x \end{array} \right] % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= \left[ \begin{array}{c} \mathbf{1} \cdot y \\ x \cdot y \end{array} \right] % \end{align} $$\Downarrow$$ \begin{align} % \beta &= \left( \mathbf{A}^{*} \, \mathbf{A} \right)^{-1} \mathbf{A}^{*} \,y \\ % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= % inv \left( \left( \mathbf{1} \cdot \mathbf{1} \right) \left( x \cdot x \right) - \left( \mathbf{1} \cdot x \right)^{2} \right)^{-1} \left[ \begin{array}{rr} x \cdot x & -\mathbf{1} \cdot x \\ -\mathbf{1} \cdot x & \mathbf{1} \cdot \mathbf{1} \end{array} \right] % \left[ \begin{array}{c} \mathbf{1} \cdot y \\ x \cdot y \end{array} \right] % \end{align}

Solution method 2: Moore-Penrose pseudoinverse: $$\beta = \mathbf{A}^{+} y$$

The MATLAB intrinsic mldivide is one option.