1
$\begingroup$

From the paper for Generalized low-rank models by Stephen Boyd, this Frobenius loss function has been used using SVD. Can someone explain it to me the following equation? Is U inverse is equal to U transpose here or what has been done here as A = U sigma V transpose? And if we make it like sigma = U inverse A V inverse or what's going on. also why it is equal to sigma - U trans XY V? Please guide me. Thanks

enter image description here

$\endgroup$
  • $\begingroup$ HINT: Say $Z = A - XY$, and $A = U\Sigma V^T$, and the Frobenius norm of $\lVert Z\rVert_F = \lVert UZV^T\rVert_F$, where $U$ and $V$ are unitary matrices, i.e., $UU^T = I = U^TU$ and $VV^T = I = V^TV$. $\endgroup$ – user550103 Jul 9 '18 at 9:11
1
$\begingroup$

There is a theorem called the Eckart-Young-Mirsky Theorem which states the following.

$$A = U \Sigma V^{T} $$ $$ A_{k} = \sum_{i=1}^{k} \sigma_{i} u_{i} v_{i}^{T} $$ $$ \| A - A_{k} \|_{2} = \| \sum_{i=k+1}^{n} \sigma_{i} u_{i} v_{i}^{T} \| = \sigma_{k+1} $$

Note

$$ \| A - A_{k} \|_{F}^{2} = \| \sum_{i=k+1}^{n} \sigma_{i} u_{i} v_{i}^{T} \|_{F}^{2} = \sum_{i=k+1}^{n} \sigma_{i}^{2} $$

Now...utilizing your stuff

$$ \| A - XY \|_{F}^{2} = \| \Sigma - U^{T} XY V \|_{F}^{2} $$ From above we have seem that because of the unitary invariance under the 2 norm that $U,V$ go away and we're left with simply the singular value matrix when approximating $A$ so that is why we have $\Sigma$ there.

If you read it define $X,Y$ neatly within the Eckart Mirsky theorem to be a low-rank approximation. We end up with

$$X = U_{k}\Sigma_{k}^{\frac{1}{2}} Y = \Sigma_{k}^{\frac{1}{2}} V_{k}^{T} $$ $$ XY = A_{k} $$ $$ \| \Sigma - U^{T} A_{k} V\|_{F}^{2} $$ But those are unitary..we'll end up with $$\| \Sigma - \Sigma_{k} \|_{F}^{2} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.