# Use least squares to estimate coefficients in a linear system containing noise

Problem
I have a linear system of the form, $y=Ax+v$, where $v$ is noise. I need to use least squares to estimate the coefficients of the matrix $A$.

Atempt
I made the assumption that the error, $v$, approaches $0$, such that I can form the problem as $Ax=y$. I then right multiplied both sides by $x^T$, resulting in $Axx^T=yx^T$. Lastly, I solved for $A$, $A=yx^T(xx^T)^{-1}$.

Upon testing my solution (with the given y and x matrices), by performing $y - Ax$, the resulting matrix should have been close to $0$, but I instead had values with magnitudes less than $10$.

What am I doing wrong / missing?

Edit
More info: $A\in\mathbb{R}^{m x n}, x\in\mathbb{R}^{n}, y\in\mathbb{R}^{m}$

-

Least squares solution is given as

$$(X^TX)^{-1}X^Ty$$

and depending on the form of $X$ and $y$ the solution can be slightly different. However there is a quadratic term $(X^TX)$ which is inverted and the cross term $X^Ty$.

According to this information, your solution seems okay. However you can not claim that $y-Ax$ should be close $0$. It should read like this;

Given all matrices $A$, the one that you chose minimizes the Mean Squared Error ($MSE$). It minimizes only... namely you can choose any other $A$ and see that that $A$ is suboptimal with respect ti $MSE$.

From the practical point of view if your $X$ has some large values then it is possible to get $y-AX$ which has values close to $10$

-
This doesn't work due to the dimensions of the matrices, see my edit. You're right about MSE, but I would still assume that for this problem the error should be much smaller. –  TehTechGuy Jan 28 '13 at 0:56
I saw. Please have a look at the following of the answer. Your solution is OK. –  Seyhmus Güngören Jan 28 '13 at 0:59
I just calculated MSE and it seemed within reason. For the future I will have to make sure not to assume the data will work out correctly. Thanks. –  TehTechGuy Jan 28 '13 at 1:11