0
$\begingroup$

I want to optimize the following equation in order to get the optimal $\beta$, where $U \in R^{m \times n}, V \in R^{m \times n},\beta \in R^{k}$.

Are three optimization equations are equal to obtain $\beta$?

First One:

It is NOT squared Frobenius norm in the 1st term.

$$\text{arg} \min_{\beta}\left\|U- \sum_{i=1}^{k} \beta_i V_i \right\|_F + \lambda\|\beta\|_2^2 $$

Second One:

It is squared Frobenius norm in 1st term compared First One.

$$\text{arg} \min_{\beta}\left\|U- \sum_{i=1}^{k} \beta_i V_i \right\|_F^{2} + \lambda\|\beta\|_2^2$$

Third One:

There is a coefficient $\frac{1}{2 \sigma n}$ which is also constant in the 1st term compared to First One.

$$\text{arg} \min_{\beta} \frac{1}{2\sigma n} \|U- \sum_{i=1}^{k} \beta_i V_i \|_F + \lambda\|\beta\|_2^2$$

I want to know if these three optimization are equal to obtain the optimal $\beta$? That is to say: the optimal $\beta$ should be same for the above three objective functions?

I understand the Second One can be equal to standard form ridge regression optimization problem as suggested by optimize weight coefficient: $\text{arg} \min_{\beta}\|U- \sum_{i=1}^{k} \beta_i V_i \|_F + \lambda\|\beta\|_2^2$

When optimizing, can I arbitrarily add or substract constant coefficients for 1st or 2nd terms, (like First one and Third one)?

Thanks.

$\endgroup$
  • 1
    $\begingroup$ No, they are not equivalent. However by choosing a different value of the parameter $\lambda$ you can make the third equivalent to the second. $\endgroup$ – Rahul Apr 24 '17 at 21:00
  • $\begingroup$ @Rahul Thanks. Can you point me out what is an appropriate algorithm to optimize Third One? $\endgroup$ – BioChemoinformatics Apr 24 '17 at 22:16
  • $\begingroup$ @Rahul Is $\text{arg} \min_{\beta} \frac{1}{2\sigma n} \|U- \sum_{i=1}^{k} \beta_i V_i \|_F^2 + \lambda\|\beta\|_2^2$ equivalent to $\text{arg} \min_{\beta} \|U- \sum_{i=1}^{k} \beta_i V_i \|_F^2 + 2 \sigma n \lambda \|\beta \|_2^2$? $\endgroup$ – BioChemoinformatics Apr 24 '17 at 22:28
  • 1
    $\begingroup$ Yes, minimizing $af(x)$ is equivalent to minimizing $f(x)$ as long as $a$ is constant. $\endgroup$ – Rahul Apr 24 '17 at 22:32
  • $\begingroup$ @Rahul Thanks. May I ask one more question: generally, squared Frobenius norm is more common than just Frobenius norm, right? $\endgroup$ – BioChemoinformatics Apr 24 '17 at 23:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.