Matrix Weighted Least Squares with Nuclear Norm Minimization

Question

Nuclear norm minimization is very popularization and formulation is least squares term with nuclear norm term as following, $$\min\limits_{X} \frac{1}{2}\|X_{3\times3}-Y_{3\times3}\|_F^2+\lambda\|X_{3\times3}\|_{*}.$$

The above minimization can be solved by SVT (singular value thresholding). And the least squares term is based on the hypothesis that the noise is Gausssian noise and i.i.d..

However if the noise is Gaussian noise with different variance, we can get the weighted least squares as following, \begin{equation} \min\limits_{X} \frac{1}{2}\|\big(X_{3\times3}-Y_{3\times3}\big)W_{}\|_F^2+\lambda\|X_{3\times 3}\|_{*} \end{equation}

where $W$ is the diagonal matrix,

$$W=\begin{pmatrix} w_1&0&0 \\0&w_2 &0\\ 0& 0 & w_3\end{pmatrix}$$

My question is how to solve the optimization problem above? Any suggestion or reference paper is appreciated.

Answer 1

If you already know how to solve the first type of problem, an obvious approach is to make the second problem look like the first.

Towards that end, let $$\eqalign{ X' & = XW \cr Y' &= YW \cr \lambda' &= \lambda \, \|W^{-1}\|_* \cr }$$

In terms of the primed variables, the second problem now looks like the first.

This transformation relies on the fact that the nuclear norm is sub-multiplicative.

Answer 2

This problem can be solved use standard proximal gradient descent algorithms, and this

• https://web.stanford.edu/~boyd/papers/prox_algs.html

is a good guidance to it. Hope it helps.