Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Problem
I have a linear system of the form, $y=Ax+v$, where $v$ is noise. I need to use least squares to estimate the coefficients of the matrix $A$.

Atempt
I made the assumption that the error, $v$, approaches $0$, such that I can form the problem as $Ax=y$. I then right multiplied both sides by $x^T$, resulting in $Axx^T=yx^T$. Lastly, I solved for $A$, $A=yx^T(xx^T)^{-1}$.

Upon testing my solution (with the given y and x matrices), by performing $y - Ax$, the resulting matrix should have been close to $0$, but I instead had values with magnitudes less than $10$.

What am I doing wrong / missing?

Edit
More info: $A\in\mathbb{R}^{m x n}, x\in\mathbb{R}^{n}, y\in\mathbb{R}^{m}$

share|improve this question

1 Answer 1

up vote 0 down vote accepted

Least squares solution is given as

$$(X^TX)^{-1}X^Ty$$

and depending on the form of $X$ and $y$ the solution can be slightly different. However there is a quadratic term $(X^TX)$ which is inverted and the cross term $X^Ty$.

According to this information, your solution seems okay. However you can not claim that $y-Ax$ should be close $0$. It should read like this;

Given all matrices $A$, the one that you chose minimizes the Mean Squared Error ($MSE$). It minimizes only... namely you can choose any other $A$ and see that that $A$ is suboptimal with respect ti $MSE$.

From the practical point of view if your $X$ has some large values then it is possible to get $y-AX$ which has values close to $10$

share|improve this answer
    
This doesn't work due to the dimensions of the matrices, see my edit. You're right about MSE, but I would still assume that for this problem the error should be much smaller. –  TehTechGuy Jan 28 '13 at 0:56
    
I saw. Please have a look at the following of the answer. Your solution is OK. –  Seyhmus Güngören Jan 28 '13 at 0:59
    
I just calculated MSE and it seemed within reason. For the future I will have to make sure not to assume the data will work out correctly. Thanks. –  TehTechGuy Jan 28 '13 at 1:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.