Proof of Inequality: smallest and largest eigenvalue bounds Given a matrix $A$, we define $\lambda_{min}(A^{T}A)$ and $\lambda_{max}(A^{T}A)$ to be the smallest
and largest eigenvalues of $A^TA$. Show that for any $x,y \in R^{n} $:
$\lambda_{min}(A^{T}A) \, \vert\vert x-y \vert\vert^{2} \leq \vert\vert A(x-y) \vert\vert^{2} \leq \lambda_{max}(A^{T}A) \, \vert\vert x-y \vert\vert^{2}$
I have managed to prove one part of the inequality:
$\vert\vert A(x-y) \vert\vert^{2} \leq \vert\vert A \vert\vert^{2}\vert\vert (x-y) \vert\vert^{2} = \sigma_{max}^2(A) \, \vert\vert (x-y) \vert\vert^{2} = \lambda_{max}(A^{T}A) \, \vert\vert x-y \vert\vert^{2} \qquad$ ($\sigma_{max}$: Largest singular value of A)
But I am not sure how to derive the second inequality. I would really appreciate any help.
 A: Usually we don't use the known fact that $\|A\| = \sigma_\max(A)$ because the inequality in question is used to prove this fact.
One can prove both sides of the inequality by noting that $A^T A$ is a positive semidefinite matrix and can hence be orthogonally diagonalized as $Q^T D Q$, for orthogonal $Q$ and $D$ having non-negative diagonal entries. WLOG, assume $\vec{v}$ has magnitude $1$. (Why can we do this?) To find the maximum and minimum value of
$$\|A \vec{v}\|^2 = \vec{v}^T A^T A \vec{v} = \vec{v}^T Q^T D Q \vec{v} = \vec{y}^T D \vec{y}$$
where $\vec{y} = Q \vec{v}$, note that if $\vec{v}$ is some arbitrary unit vector, then so too is $\vec{y}$ (as $Q$ is an orthogonal transform). So it suffices to maximize (and minimize)
$$\vec{y}^T D \vec{y} = \sum_{i = 1}^n d_i y_i^2, ~~~ \text{ subject to } \sum_{i = 1}^n y_i^2 = 1$$
where $d_i$ are the individual diagonal entries from top to bottom. Can you show why the minimum and maximum of this expression are $\lambda_\min(A^T A) = \min d_i$ and $\lambda_\max(A^T A) = \max d_i$, respectively?
A: More compact: set $z=x-y$ you want to prove
$$\lambda_{\min}(A^TA)\le \langle A^TA\frac{z}{\|z\|},\frac{z}{\|z\|}\rangle \le \lambda_{\max}(A^TA)$$
By diagonalizing $A^TA$; for a unitary $U$
$\langle A^TAU\dfrac{z}{\|z\|},U\dfrac{z}{\|z\|}\rangle =\langle U^*A^TAU\dfrac{z}{\|z\|},\dfrac{z}{\|z\|}\rangle=v^*Dv$
where $D$ is the diagonal eigenvalue matrix of $A^TA$ and $v=\dfrac{z}{\|z\|}$
$v^*Dv$ is a convex combination of the eigenvalues so inequality follows.
