Proof of an inequality for quadratic forms I'm confused at the proof for the inequality $x^TAx \leq λ_{i}x^Tx$ assuming the matrix A is symmetric and eigenvalues are sorted so $λ_{1} \geq ... \geq λ_{n}$
$$x^TAx = x^TQDQ^Tx = (Q^Tx)^TD(Q^Tx) = \sum_{i = 1}^{n}λ_{i}(q_{i}^Tx)^2 \leq λ_{i}\sum_{i = 1}^{n}(q_{i}^Tx)^2 = λ_{i}||x||^2$$
Specifically $(Q^Tx)^TD(Q^Tx) = \sum_{i = 1}^{n}λ_{i}(q_{i}^Tx)^2$, I understand where $(q_{i}^Tx)^2$ comes from as we're summing out over all the $q_{i}$ but not why there's a $λ_{i}$ or how we can rewrite the diagonal matrix as that.
And at the end, it's concluded that $\sum_{i = 1}^{n}(q_{i}^Tx)^2 = λ_{i}||x||^2$ but I'm unsure of how the sum over $(q_{i}^Tx)^2$ is the same as the norm of $x$. I understand $x^Tx$ is the norm squared but not why $q_{i}^Tx$ is the same.
Is this proof correct (I'd hope so considering the source) or am I misunderstanding something? Would appreciate help.
 A: Let me do it without the matrices because I am horrible working with them. You have a transformation $A:\mathbb R^n \to \mathbb R^n$. Since $A$ is symmetric there is a orthonormal basis $b_1,\ldots,b_n$ of $\mathbb R^n$ such that $Ab_j = \lambda_j b_j$, where $\lambda_j$ is a eigenvalues. We can suppose $\lambda_1\geq \cdots \geq \lambda_n$ because we can reindex the $b_j$.
The expression $x^TAx$ is the inner product $\langle x,Ax\rangle$ in matrix language. Lets calculate $\langle x,Ax\rangle$.  Since the basis $b_j$ is orthonormal we have $$x = \sum_j \langle x,b_j\rangle b_j$$
and therefore we can write
$$Ax = \sum_j \langle x,b_j\rangle Ab_j = \sum_j \lambda_j\langle x,b_j\rangle b_j.$$
Therefore we obtain:
$$\langle x,Ax \rangle = \sum_{i,j} \langle x,b_i\rangle \lambda_j \langle x,  b_j\rangle \langle b_i,b_j\rangle =\sum_k \lambda_k |\langle x,b_k \rangle|^2.$$
Now we use that $\lambda_1 \geq \lambda_k$ and conclude that $$\langle x, Ax\rangle =\sum_k \lambda_k |\langle x,b_k \rangle|^2\leq\lambda_1 \sum_k  |\langle x,b_k \rangle|^2.$$
Just calculate $\langle x,x \rangle$ and you will obtain
$$\Vert x \Vert^2 =\langle x,x \rangle = \sum_k |\langle x,b_k \rangle|^2$$
and therefore we conclude that
$$\langle x,Ax \rangle \leq \lambda_1 \Vert x\Vert^2.$$ 
A: There are two fundamental steps in the proof:


*

*the first relies on $A$ being real and symmetric, thus admitting a diagonalization, $Q^T A Q = \Lambda$, through a change of coordinates $z=Q^Tx$ with $Q$ being an orthogonal matrix (satisfying $Q^{-1} = Q^T$);

*being $Q$ an orthogonal matrix, it preserves distance in the transformation ( isometry), that is
$$
\|z\|_2 = \|Q^T x\|_2 = \|x\|_2  \qquad (1)
$$
With this in mind, the last (unclear) step in the proof where $q_i$ denotes the $i$-th column of $Q$ (right eigenvectors) just restates (1)
$$
\|Q^Tx\|_2^2 = \sum_{i=1}^n \left(q_i^T x \right)^2 = \|x\|_2^2
$$
