I would like to optimize the follow objective function, where $U \in \mathbb{R}^{m \times n}$, $V_i \in \mathbb{R}^{m \times n}$, $\beta \in \mathbb{R}^{k}$.

Actually, $\beta$ is the weight of each $V$, what is the algorithm to get the optimal $\beta$? Here the first term is Frobenius norm and second term is $2$-norm, I am not familiar with this kinds of objective function, can someone help me understand this objective function? what existing software (R or Matlab) can solve this? or need code by myself? Is this similar to ridge regression or lasso or others?


$$\text{arg} \min_{\beta}\left\|U- \sum_{i=1}^{k} \beta_i V_i \right\|_F + \lambda\|\beta\|_2^2$$

  • 1
    $\begingroup$ This is ridge regression. You can see this by rearranging $U$ and $V_i$'s as $mn \times 1$ vectors by stacking columns. $\endgroup$ – Arin Chaudhuri Apr 23 '17 at 2:14
  • $\begingroup$ Thanks. I will try to understand this. Find the solver to solve ridge regression. $\endgroup$ – BioChemoinformatics Apr 23 '17 at 18:03
  • $\begingroup$ @ArinChaudhuri I find that for ridge regression, the first term should be squared Frobenius. But my optimization function is NOT squared, how to deal with this? Thanks. $\endgroup$ – BioChemoinformatics Apr 24 '17 at 22:22
  • $\begingroup$ You are correct, I missed that. I think they are still equivalent in the sense that the solution with $\| \|_{F}$ for a given $\lambda$ is the same as solution with $\| \|^2_{F}$ with another $\lambda$. $\endgroup$ – Arin Chaudhuri Apr 24 '17 at 23:22
  • $\begingroup$ Thanks @ArinChaudhuri. It seems that squared Frobenius norm is more common than not squared one in cost function. So the solution for an optimization problem depends on the user's choice, right. But I can select the squared Frobenius norm in the first term instead of NOT squared one. It should also be OK. Please give some suggestions. Thanks. $\endgroup$ – BioChemoinformatics Apr 24 '17 at 23:59

First, change $U$ to $R^{mn \times 1}$, called $y$, and for each of $V_1,...,V_k$ also change to $R^{mn \times 1}$ and column bind to form a new matrix, say $X$.

then the queation is ridge regression problem and the solution for $beta$ is:

$$(X^TX + \lambda I)^{-1} X^T y $$

thanks @Arin Chaudhuri.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.