I have a problem where the regularizer is the nuclear norm and the matrix being regularized is $n \times d$ with $d < n$. I was initially not getting low rank for the desired performance, the matrix was still at rank $d$. When the rank started reducing from $d$, the performance was not good enough.

So, I started trying Schatten norm with $p = 0.5$. However, now I am getting a regularization path where there is suddenly a huge jump in the rank from $d$ to a low value (like $4$). Naturally, when the jump to the (much lower) rank happens, the performance is very poor. I tried scaling the loss part, to see whether it will stretch the regularization path. But the steep jump in the rank continues to happen.

Ideally, I want an optimal point where the rank of the matrix has reduced from $d$ and the desired performance is achieved at that point. I am a little surprised that low rank regularization has such practical difficulties.

  • 2
    $\begingroup$ You haven't provided any background on the problem that you're trying to solve, so its unlikely that you'll get any particularly useful answers. Please fill in the background. $\endgroup$ – Brian Borchers Apr 25 '18 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.