# least-squares problem with matrix

Consider the samples of vectors $(x_0,x_1,...,x_k)$ where $x\in \mathbb{R}^m$ and $(y_0,y_1,...,y_k)$ where $y\in \mathbb{R}^n$.
I need to find the matrix $A\in \mathbb{R}^{n\times m}$ of the following least-squares problem :

$\min_A S= \sum\limits_{i=0}^k (y_i-Ax_i)^T(y_i-Ax_i)$

To find the minimum, I differentiate $\frac{\partial S}{\partial A}=0=\sum\limits_{i=0}^k -2x_i^Ty_i +2x_i^TAx_i$

And finally : $\sum\limits_{i=0}^k x_i^TAx_i=\sum\limits_{i=0}^k x_i^Ty_i$

I am stuck here, how to find $A$ ?

• Maybe that could help someone, the problem can be computed by a LAPACK routine : ?GELSS – Michael M. Sep 28 '15 at 8:59

You can rewrite it as $$\sum\limits_{i=0}^k (y_i-Ax_i)^T(y_i-Ax_i)=\sum\limits_{i=0}^k ||y_i-Ax_i||_2^2=||Y-AX||_F^2$$ where X and Y are the matrices of the samples by column, and the $||\cdot||_F$ is the Frobenius norm. In this way you can transpose everything $$\dots=||Y^T-X^TA^T||_F^2$$ and you can treat it as a least squares minimization problem, that can be solved exploiting the pseudoinverse matrix. Remember that here $X^T,Y^T$ and $A^T$ play the roles respectively of the matrix of the linear system, constant term (one vector for each column) and the unknown (one vector for each column).
• So the final result is $A=((X^T)^+Y^T)^T$ ? – Michael M. Sep 28 '15 at 7:33
• Yes, usually this is the way. Just remember that you can pseudoinvert a matrix $D\in\mathbb{R}^{r\times c}$ with $r\geq c$ if $D$ is full-rank. In fact you need to invert the matrix $D^T D$. For you the matrix $D=X^T$, so you need to check these hypotheses. If your matrix mismatch the hypothesis of $r\geq c$ than your system is underdeterminated, so you loose the uniqueness of the solution. – Ilis Sep 28 '15 at 8:33