Relationship between Spectral Decomposition, Positive Definite Matrix and Quadratic Form

I am studying Linear Algebra. I have faced a problem to understand Symmetric Matrix with spectral decomposition. After I studied spectral decomposition, the next page in my book talks about a positive definite matrix and quadratic form.

I am kind of lost what relationships are there between symmetric decomposition, a positive definite matrix, and quadratic form.

Hope I can have some explanations. Thank you in advance.

1 Answer

The connection is that every quadratic form can be written in the form $$Q(x) = x^TAx$$ for the correct real symmetric matrix $A$. For example, for quadratic forms on $2$ variables, we can write $$Q(x_1,x_2) = ax_1^2 + bx_1x_2 + cx_2^2 = \\ \pmatrix{x_1&x_2} \pmatrix{a&b/2\\b/2&c} \pmatrix{x_1\\x_2}$$ Spectral decompositions give us a "change of variables" that make quadratic forms easier to understand.

In particular, suppose that we have $Q(x) = x^TAx$. $A$ has spectral decomposition $A = UDU^T$ for orthogonal matrix $U$ and diagonal matrix $D$ whose diagonal entries are $\lambda_1,\dots,\lambda_n$. We now have $$Q(x) = x^T U DU^Tx = (U^Tx)^T D (U^Tx)$$ that is, if we make the substitution $y = U^Tx$ (which is to say $x = Uy$) and define the simpler quadratic form $Q'(x) = x^TDx$, we have $$Q(x) = Q'(y) = y^TDy = \lambda_1 y_1^2 + \cdots + \lambda_n y_n^2$$ For example, an important consequence we can gather is Rayleigh's theorem, which says that if $\lambda_1$ is the lowest eigenvalue, then the expression $Q(x)/\|x\|^2$ is minimized when $y_2,\dots,y_{n} = 0$.

• Here,what does the term, a positive definite matrix, mean? Apr 9 '15 at 2:36
• Positive definite means it's symmetric and the eigenvalues $\lambda_i$ are positive. Apr 9 '15 at 2:46
• Does that mean symmetric matrices always produces positive eigenvalues , therefore they are always positive definite matrices? Or symmetric matrices are positive when their eigenvalues are all positive? Apr 9 '15 at 2:50
• The second one. Apr 9 '15 at 8:31