Proof that the eigenvectors of the matrix $A$ maximizes the scalar $v^TAv$ I have been reading up on finding the eigenvectors and eigenvalues of a symmetric matrix lately and I am totally unsure of how and why it works. Given a matrix, I can find its eigenvectors and values like a machine but the problem is, I have no intuition of how it works.
1) I understand that $v^tAv$ is the equation of an ellipse in matrix form
2) I understand how Lagrangian multipliers work
Can someone please show me the proof where finding the eigenvectors of such a matrix gives me the principal components ? 
I am following this topic.
Maximizing symmetric matrices v.s. non-symmetric matrices I know how to find the eigenvalues of the matrix but I have no idea how it works
 A: Symmetric matrices can always be orthogonally diagonalised.
That is to say, $A=P^TDP$ for some orthogonal matrix $P$ and diagonal matrix $D$. The columns of $P^T$ are eigenvectors of $A$.
As such, $v^TAv=v^TP^TDPv=(Pv)^TD(Pv)$.
Let $D=\left(\begin{matrix}
\lambda_1&0&0&\cdots&0&0&0\\
0&\lambda_2&0&\cdots&0&0&0\\
0&0&\lambda_3&\cdots&0&0&0\\
\vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\
0&0&0&\cdots&\lambda_{n-2}&0&0\\
0&0&0&\cdots&0&\lambda_{n-1}&0\\
0&0&0&\cdots&0&0&\lambda_n\\
\end{matrix}\right)$, and $Pv=a_1e_1+a_2e_2+\dots+a_ne_n$, where $e_i$ is the standard basis.
Then $(Pv)^TDPv=\lambda_1a_1^2+\lambda_2a_2^2++\lambda_3a_3^2+\dots+\lambda_na_n^2$.
By "eigenvectors", I would assume you meant eigenvectors which are of length $1$, otherwise we can arbitrary increase the length of $v$ and increase the magnitude of $v^TAv$.
As $P$ is an orthogonal matrix, $Pv$ has length $1$, $a_1^2+a_2^2+a_3^2+\dots+a_n^2=1$.
$(Pv)^TDPv=\lambda_1a_1^2+\lambda_2a_2^2++\lambda_3a_3^2+\dots+\lambda_na_n^2\leq\lambda_{max}(a_1^2+a_2^2+a_3^2+\dots+a_n^2)=\lambda_{max}$.
Equality occurs when $\lambda_i\neq\lambda_{max},a_i=0$.
A special case of equality is when $a_i=1$ for only $1$ value $i$. This is obtained then $v$ is the $i$th column of $P^T$, so $v$ is an eigenvector of $A$.
A: The eigenvectors $v_1,v_2,\dots,v_n$ of a symmetric matrix form an orthonormal basis for ${\bf R}^n$. So any vector $v$ can be expressed as a linear combination, $$v=c_1v_1+c_2v_2+\cdots+c_nv_n$$ Then you can calculate $$v^tAv=\lambda_1c_1^2+\lambda_2c_2^2+\cdots+\lambda_nc_n^2$$ where $\lambda_i$ is the eigenvalue for the eigenvector $v_i$, $i=1,2,\dots,n$. 
Now you should be able to see why, if (say) $\lambda_1$ is the biggest eigenvalue, then $v=v_1$ maximizes $v^tAv$ among all unit vectors. 
