Any one can help give examples of applications when a matrix ${\bf A}$ in the linear system ${\bf A}{\bf x} = {\bf b}$ is Hermitian positive definite?

Or another way of asking, why such linear system is useful, and any example cases?


Because of my statistics background, I can think of covariance matrix or correlation matrix as positive semi-definite, but currently I have no idea if they can be involved in a linear system.

In some materials I read about numerical method solving linear system, the method/proofs are specifically targeting the Hermitian positive definite matrix, and I want to know a little bit about "why". The following is an example (link to book). Thanks!

enter image description here

  • $\begingroup$ What do you mean? Any hermitian matrix can be "involved" in a linear system... $\endgroup$ – amsmath Oct 8 '17 at 1:04
  • $\begingroup$ @amsmath Thanks for your advise. I rephrased the question. I am asking why such linear system is useful. $\endgroup$ – Tony Oct 8 '17 at 1:08
  • $\begingroup$ A covariance matrix that is positive semidefinite but not positive definite describes a degenerate distribution. There are significant complications that result from considering such distributions and this is not often done in practice. $\endgroup$ – Brian Borchers Oct 8 '17 at 1:16

Some common ways in which symmetric and positive definite matrices appear in linear systems of equations.

  1. One way of solving the linear least squares problem $\min \| Ax-b \|_{2}^{2}$ (where $A$ doesn't need to be symmetric or even square) is to solve the normal equations $A^{T}Ax=A^{T}b$. It can be shown that if $A$ has full column rank, then $A^{T}A$ will be symmetric and positive definite.

  2. In many cases, the linear systems of equations resulting from the discretization of a partial differential equation are symmetric and positive definite. This happens for example if you discretize Poisson's equation using finite differences on a regular square grid.

  3. In Newton's method applied to minimize a smooth strictly convex function, the Hessian matrix will be symmetric and positive definite.

  4. In statistics, covariance matrices $\Sigma$ are required to be symmetric and positive definite. There are some situations in which you need to solve systems of equations involving a symmetric and positive definite covariance matrix.

Although many textbooks on iterative methods for linear systems of equations start with the symmetric and positive definite case, many methods have been developed to solve linear systems of equations and least squares problems where $A$ is not symmetric and positive definite.

Keep in mind that in the world of direct factorization methods for linear systems of equations both Cholesky (for symmetric and PD matrices) and LU factorization (for other matrices) are used.

In both worlds (iterative methods and direct factorization methods) it's wise to make use of the symmetric and positive definite properties of your system of equations if possible.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.