1
$\begingroup$

Consider the samples of vectors $(x_0,x_1,...,x_k)$ where $x\in \mathbb{R}^m$ and $(y_0,y_1,...,y_k)$ where $y\in \mathbb{R}^n$.
I need to find the matrix $A\in \mathbb{R}^{n\times m}$ of the following least-squares problem :

$\min_A S= \sum\limits_{i=0}^k (y_i-Ax_i)^T(y_i-Ax_i)$

To find the minimum, I differentiate $\frac{\partial S}{\partial A}=0=\sum\limits_{i=0}^k -2x_i^Ty_i +2x_i^TAx_i$

And finally : $\sum\limits_{i=0}^k x_i^TAx_i=\sum\limits_{i=0}^k x_i^Ty_i$

I am stuck here, how to find $A$ ?

$\endgroup$
  • $\begingroup$ Maybe that could help someone, the problem can be computed by a LAPACK routine : ?GELSS $\endgroup$ – Michael M. Sep 28 '15 at 8:59
1
$\begingroup$

You can rewrite it as $$\sum\limits_{i=0}^k (y_i-Ax_i)^T(y_i-Ax_i)=\sum\limits_{i=0}^k ||y_i-Ax_i||_2^2=||Y-AX||_F^2$$ where X and Y are the matrices of the samples by column, and the $||\cdot||_F$ is the Frobenius norm. In this way you can transpose everything $$\dots=||Y^T-X^TA^T||_F^2$$ and you can treat it as a least squares minimization problem, that can be solved exploiting the pseudoinverse matrix. Remember that here $X^T,Y^T$ and $A^T$ play the roles respectively of the matrix of the linear system, constant term (one vector for each column) and the unknown (one vector for each column).

$\endgroup$
  • $\begingroup$ So the final result is $A=((X^T)^+Y^T)^T$ ? $\endgroup$ – Michael M. Sep 28 '15 at 7:33
  • $\begingroup$ Yes, usually this is the way. Just remember that you can pseudoinvert a matrix $D\in\mathbb{R}^{r\times c}$ with $r\geq c$ if $D$ is full-rank. In fact you need to invert the matrix $D^T D$. For you the matrix $D=X^T$, so you need to check these hypotheses. If your matrix mismatch the hypothesis of $r\geq c$ than your system is underdeterminated, so you loose the uniqueness of the solution. $\endgroup$ – Ilis Sep 28 '15 at 8:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.