I am studying Linear Algebra. I have faced a problem to understand Symmetric Matrix with spectral decomposition. After I studied spectral decomposition, the next page in my book talks about a positive definite matrix and quadratic form.

I am kind of lost what relationships are there between symmetric decomposition, a positive definite matrix, and quadratic form.

Hope I can have some explanations. Thank you in advance.


The connection is that every quadratic form can be written in the form $$ Q(x) = x^TAx $$ for the correct real symmetric matrix $A$. For example, for quadratic forms on $2$ variables, we can write $$ Q(x_1,x_2) = ax_1^2 + bx_1x_2 + cx_2^2 = \\ \pmatrix{x_1&x_2} \pmatrix{a&b/2\\b/2&c} \pmatrix{x_1\\x_2} $$ Spectral decompositions give us a "change of variables" that make quadratic forms easier to understand.

In particular, suppose that we have $Q(x) = x^TAx$. $A$ has spectral decomposition $A = UDU^T$ for orthogonal matrix $U$ and diagonal matrix $D$ whose diagonal entries are $\lambda_1,\dots,\lambda_n$. We now have $$ Q(x) = x^T U DU^Tx = (U^Tx)^T D (U^Tx) $$ that is, if we make the substitution $y = U^Tx$ (which is to say $x = Uy$) and define the simpler quadratic form $Q'(x) = x^TDx$, we have $$ Q(x) = Q'(y) = y^TDy = \lambda_1 y_1^2 + \cdots + \lambda_n y_n^2 $$ For example, an important consequence we can gather is Rayleigh's theorem, which says that if $\lambda_1$ is the lowest eigenvalue, then the expression $Q(x)/\|x\|^2$ is minimized when $y_2,\dots,y_{n} = 0$.

  • $\begingroup$ Here,what does the term, a positive definite matrix, mean? $\endgroup$
    – user122358
    Apr 9 '15 at 2:36
  • $\begingroup$ Positive definite means it's symmetric and the eigenvalues $\lambda_i$ are positive. $\endgroup$ Apr 9 '15 at 2:46
  • $\begingroup$ Does that mean symmetric matrices always produces positive eigenvalues , therefore they are always positive definite matrices? Or symmetric matrices are positive when their eigenvalues are all positive? $\endgroup$
    – user122358
    Apr 9 '15 at 2:50
  • $\begingroup$ The second one. $\endgroup$ Apr 9 '15 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.