0
$\begingroup$

i have the following two general convex optimization problems: $$\min_X \sum_i f_i(x_i,D)$$ $$\min_{D.W} \sum_i g_i(x_i,W,D)$$ $X,D,W$ are parameter matrices and $Y$ is the data samples matrix. $f,g$ are convex functions and deferentiable respect to the above matrices.

As you see these problems are coupled and the solution to one can effect the other. For example finding optimum $X$ for the first problem provides different set of $g_i$ functions (for instance imagine $g_i=x_iWD$).

I like to find the optimum point $X^*,D^*,W^*$ for the whole problem. However the 2nd optimization is the main goal of the problem, so maybe we can call it a bilevel optimization, but not sure about it!

I tried a solution as follow:

1- Finding best $X^*$ matrix based on the first problem.

2- Finding best $W^*$ matrix based on the 2nd problem.

3- Finding best $D^*$ matrix based on the 3rd problem.

and repeating 1-3 until convergence. The problem is that the result is sub-optimal as i tried it with toy-data and it failed to find the better optimum point that exist.

also i'm not sure if the whole strategy is properly chosen to solve the above problem.

$\endgroup$
  • 1
    $\begingroup$ So which solution are you looking for? Minimal sum of the two objectives, minimal product, just some kind of fixed pont etc. The problem is not well defined $\endgroup$ – Johan Löfberg Jul 19 '17 at 19:11
  • 1
    $\begingroup$ Or is it a bilevel program you want to solve where $X$ is the inner variable and you want $X$ to be $\argmin$ of the first objective, and then you want to minimize the second (the order is important, $D$ and $W$ would then be the master variables and $X$ the follower) $\endgroup$ – Johan Löfberg Jul 19 '17 at 19:16
  • $\begingroup$ You refer to a data samples matrix $Y$ that I do not see anywhere, and you have $x_i$ in the objective functions that seems not to be defined anywhere. Are they related? $\endgroup$ – prubin Jul 19 '17 at 20:07
  • $\begingroup$ The 2nd optimization is the main goal of the problem, however the first one is important as the internal optimization. So maybe we can call it a bi-level optimization, but not sure about it! $\endgroup$ – Babak Jul 19 '17 at 23:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.