The relationship between singular values of two matrices If $A,B\in \mathbb{R}^{m\times n}$ with $m\geq n$, assume singular values of $A$ are $\sigma_1\ge \sigma_2\ge \cdots\sigma_n;$ the singular values of $A+B$ are $\hat{\sigma}_1\ge \hat{\sigma}_2\ge \cdots\hat{\sigma}_n$.
Show that $|\sigma_i-\hat{\sigma}_i|\leq ||B||_2$, where $||B||_2=\sqrt{\rho(B^TB)}$. 
I only know that if $A,B$ are symmetric and $\sigma_i,\hat{\sigma}_i$ are defined as eigenvalues of $A$ and $A+B$, it can be done. Here I tried to compute the eigenvalue of $(A+B)^T(A+B)$ and $A^TA$, I don't know how to deal with the term $A^TB+B^TA$.
 A: By Courant Fisher Theorem: $$\sigma_i=\sqrt{\max_{\mathbb{R}^i}\min_{v\in\mathbb{R}^i} \frac{v^TA^TAv}{v^Tv}}=\max_{\mathbb{R}^i}\min_{v\in\mathbb{R}^i} \sqrt{\frac{||Av||_2^2}{||v||_2^2}}=\max_{\mathbb{R}^i}\min_{v\in\mathbb{R}^i}\frac{||Av||_2}{||v||_2} $$ 
$$\sigma_i=\sqrt{\min_{\mathbb{R}^{n+1-i}}\max_{v\in\mathbb{R}^{n+1-i}} \frac{v^TA^TAv}{v^Tv}}=\min_{\mathbb{R}^{n+1-i}}\max_{v\in\mathbb{R}^{n+1-i}}\sqrt{\frac{||Av||_2^2}{||v||_2^2}}=\min_{\mathbb{R}^{n+1-i}}\max_{v\in\mathbb{R}^{n+1-i}}\frac{||Av||_2}{||v||_2} $$
$$\hat{\sigma}_i=\sqrt{\max_{\mathbb{R}^i}\min_{v\in\mathbb{R}^i} \frac{v^T(A+B)^T(A+B)v}{v^Tv}}=\max_{\mathbb{R}^i}\min_{v\in\mathbb{R}^i} \frac{||(A+B)v||_2}{||v||_2}$$
$$\hat{\sigma}_i=\sqrt{\min_{\mathbb{R}^{n+1-i}}\max_{v\in\mathbb{R}^{n+1-i}} \frac{v^T(A+B)^T(A+B)v}{v^Tv}}=\min_{\mathbb{R}^{n+1-i}}\max_{v\in\mathbb{R}^{n+1-i}} \frac{||(A+B)v||_2}{||v||_2}$$
Because $$\min_{v\in\mathbb{R}^i}\frac{||(A+B)v||_2}{||v||_2}\ge \min_{v\in\mathbb{R}^i}\frac{||Av||_2}{||v||_2} -\max_{v\in \mathbb{R}^i}\frac{||Bv||_2}{v^Tv}\ge\min_{v\in\mathbb{R}^i}\frac{||Av||_2}{||v||_2} -||B||_2$$
$$\max_{v\in\mathbb{R}^{n+1-i}}\frac{||(A+B)v||_2}{||v||_2}\leq \max_{v\in\mathbb{R}^{n+1-i}}\frac{||Av||_2}{||v||_2} +\max_{v\in\mathbb{R}^{n+1-i}}\frac{||Bv||_2}{v^Tv}\leq\max_{v\in\mathbb{R}^{n+1-i}}\frac{||Av||_2}{||v||_2} +||B||_2$$
So we get $$\hat{\sigma}_i=\max_{\mathbb{R}^{i}}\min_{v\in\mathbb{R}^i}\frac{||(A+B)v||_2}{||v||_2}\ge \max_{\mathbb{R}^{i}}\min_{v\in\mathbb{R}^i}\frac{||Av||_2}{||v||_2} -||B||_2=\sigma_i-||B||_2$$
$$\hat{\sigma}_i=\min_{\mathbb{R}^{n+1-i}}\max_{v\in\mathbb{R}^{n+1-i}}\frac{||(A+B)v||_2}{||v||_2}\leq \min_{\mathbb{R}^{n+1-i}}\max_{v\in\mathbb{R}^{n+1-i}}\frac{||Av||_2}{||v||_2} +||B||_2=\sigma_i+||B||_2$$
Then we conlude that $|\hat{\sigma}_i-\sigma_i|\leq ||B||_2,\quad i=1,2,\cdots,n.$
