Comparison of Norm of Positive definite Matrices If $A \leq B$, and A and B are symmetric matrices, can we say that $ \| A \| \leq \|B\|$? In other words, If $A-B$ is negative semi-definite, does $ \| A \| \leq \|B\|$ hold?
 A: This is obviously false, as $-I\preceq0$ but $\|-I\|>\|0\|$ for every norm.
There are counterexamples even if you require $A$ to be positive semidefinite. E.g. consider
$$
A=\pmatrix{2&1\\ 1&1},
\ B=\pmatrix{3\\ &3},
\ D=\pmatrix{1\\ &4},
\ \|M\|=\|D^{-1}MD\|_\infty\tag{1}
$$
where $\|M\|_\infty=\max_{\|v\|_\infty=1}\|Mv\|_\infty=\max_i\sum_j|m_{ij}|$ is the induced maximum norm. Then the norm $\|\cdot\|$ in $(1)$ is a matrix norm induced by the vector norm $\|x\|=\|D^{-1}x\|_\infty$. Now $0\prec A\prec B$ but $\|A\|=6>3=\|B\|$.
However, the inequality in question is true if $A$ is positive semidefinite and the matrix norm in use is unitarily invariant. (The induced $2$-norm, Frobenius norm and all Schatten norms fall into this category.)
Unitarily invariant norms can be characterised as symmetric gauge functions of singular values. When $A\preceq B$, the Courant-Fischer minimax principle dictates that $\lambda_i(A)\le\lambda_i(B)$ when the eigenvalues of $A$ and $B$ are arranged in the same (ascending or descending) order. So, when $A$ is also positive semidefinite, we have $\sigma_i(A)\le\sigma_i(B)$ for each $i$. Hence $f\left(\sigma_1(A),\ldots,\sigma_n(A)\right)\le f\left(\sigma_1(B),\ldots,\sigma_n(B)\right)$ for every symmetric gauge function $f$. In turn, $\|A\|\le\|B\|$ when the norm is unitarily invariant.
