1
$\begingroup$

I have a problem where I need to find the least squares regression line. I have found $\beta_0$ and $\beta_1$ in the following equation

$$y = \beta_0 + \beta_1 \cdot x + \epsilon$$

So I have both the vectors $y$ and $x$.

I know that $\hat{y}$ the vector predictor of $y$ is $x \cdot \beta$ and that the residual vector is $\epsilon = y - \hat{y}$.

I know also that the least squares regression line looks something like this $$\hat{y} = a + b \cdot x$$ and that what I need to find is $a$ and $b$, but I don't know exactly how to do it. Currently I am using Matlab, and I need to do it in Matlab. Any idea how should I proceed, based on the fact that I am using Matlab?

Correct me if I did/said something wrong anyway.

$\endgroup$
  • $\begingroup$ I am not at all familiar with Matlab. However, do you know the formulas for the least-squares estimates of $a$ and $b$? $\endgroup$ – Clarinetist Dec 2 '15 at 18:15
  • $\begingroup$ @Clarinetist Are you talking about the formulas for $b_1$ and $b_2$ appearing here: ch.mathworks.com/help/curvefit/least-squares-fitting.html? $\endgroup$ – nbro Dec 2 '15 at 18:22
1
$\begingroup$

First define

X = [ones(size(x)) x];

then type

regress(y,X)

Observations:

  • the first step is to include a constant in the regression (otherwise you would be imposing $a=0$).

  • the output will be a vector with the OLS estimates $(a,b)$.

$\endgroup$
  • $\begingroup$ You mean something like this: X = [ones(length(x), 1), x]; ? $\endgroup$ – nbro Dec 2 '15 at 18:25
  • $\begingroup$ @nbro That will achieve the same goal. $\endgroup$ – mzp Dec 2 '15 at 18:27
  • $\begingroup$ Yes, I understood that the output should be a vector (a, b) in this case. But, by using the formulas for $b_1$ and $b_2$ here I would then find the same results, right? $\endgroup$ – nbro Dec 2 '15 at 18:29
  • $\begingroup$ @nbro Yes, that is correct. $\endgroup$ – mzp Dec 2 '15 at 18:31
  • $\begingroup$ But in the case I would like to use those equations I would need to find $\beta$ first, i.e. the coefficients of the real line in order to defined $\hat{y}$ (as I did), right? $\endgroup$ – nbro Dec 2 '15 at 18:34
0
$\begingroup$

Sequence of $m$ measurements: $$\left\{ x_{k}, y_{k} \right\}_{k=1}^{m}$$ Model: $$ y(x) = \beta_{0} + \beta_{1} x $$ Linear system: $$ \begin{align} % \mathbf{A} \, \beta &= y \\ % A \left[ \begin{array}{cc} 1 & x_{1} \\ 1 & x_{2} \\ \vdots & \vdots \\ 1 & x_{m} \end{array} \right] % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= \left[ \begin{array}{c} y_{1} \\ y_{2} \\ \vdots \\ y_{m} \end{array} \right] % \end{align} $$ Least squares solution: $$ \beta_{LS} = \left\{ \beta \in \mathbb{C}^{2} \colon \lVert \mathbf{A} \,x - y \rVert_{2}^{2} \text{ is minimized} \right\} $$ Solution type: we have full column rank. Solution is unique - a point.


Solution method 1: Normal equations $$ \begin{align} % \mathbf{A}^{*} \,\mathbf{A} \, \beta &= \mathbf{A}^{*} \,y \\ % A \left[ \begin{array}{cc} \mathbf{1} \cdot \mathbf{1} & \mathbf{1} \cdot x \\ x \cdot \mathbf{1} & x \cdot x \end{array} \right] % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= \left[ \begin{array}{c} \mathbf{1} \cdot y \\ x \cdot y \end{array} \right] % \end{align} $$ $$ \Downarrow $$ $$ \begin{align} % \beta &= \left( \mathbf{A}^{*} \, \mathbf{A} \right)^{-1} \mathbf{A}^{*} \,y \\ % beta \left[ \begin{array}{cc} \beta_{1} \\ \beta_{2} \end{array} \right] % &= % inv \left( \left( \mathbf{1} \cdot \mathbf{1} \right) \left( x \cdot x \right) - \left( \mathbf{1} \cdot x \right)^{2} \right)^{-1} \left[ \begin{array}{rr} x \cdot x & -\mathbf{1} \cdot x \\ -\mathbf{1} \cdot x & \mathbf{1} \cdot \mathbf{1} \end{array} \right] % \left[ \begin{array}{c} \mathbf{1} \cdot y \\ x \cdot y \end{array} \right] % \end{align} $$

Solution method 2: Moore-Penrose pseudoinverse: $$ \beta = \mathbf{A}^{+} y $$


The MATLAB intrinsic mldivide is one option.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.