Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

What is an intuitive explanation of a positive-semidefinite matrix? Or a simple example which gives more intuition for it rather than the bare definition. Say $x$ is some vector in space and $M$ is some operation on vectors.

The definition is:

A $n$ × $n$ Hermitian matrix M is called positive-semidefinite if

$$x^{*} M x \geq 0$$

for all $x \in \mathbb{C}^n$ (or, all $x \in \mathbb{R}^n$ for the real matrix), where $x^*$ is the conjugate transpose of $x$.

share|improve this question
    
en.wikipedia.org/wiki/Positive-definite_matrix have you seen this –  anonymous Nov 10 '10 at 20:20
1  
What's not intuitive that requires explanation? –  Qiaochu Yuan Nov 10 '10 at 20:20
    
Here I mean intuitive in a vector-geometry sense. The definition is very straightforward, but I'm trying to find out the motivation for its definition, etc. –  user915 Nov 10 '10 at 20:37

4 Answers 4

up vote 37 down vote accepted

One intuitive definition is as follows. Multiply any vector with a positive definite matrix. The angle between the original vector and the resultant vector will always be less than $\frac{\pi}{2}$. The positive definite matrix tries to keep the vector within a certain half space containing the vector. This is analogous to what a positive number does to a real variable. Multiply it and it only stretches or contracts the number but never reflects it about the origin.

share|improve this answer
    
Short and concise answer, thank you! As soon as I get some points I'll upvote. –  user915 Nov 10 '10 at 20:56

First I'll tell you how I think about Hermitian positive-definite matrices. A Hermitian positive-definite matrix $M$ defines a sesquilinear inner product $\langle Mv, w \rangle = \langle v, Mw \rangle$, and in fact every inner product on a finite-dimensional inner product space $V$ has this form. In other words it is a way of computing angles between vectors, or a way of projecting vectors onto other vectors; over the real numbers it is the key ingredient to doing Euclidean geometry. An inner product can be recovered from the norm $\langle Mv, v \rangle = \langle v, Mv \rangle$ it induces, and a norm in turn can be recovered from its unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$. This unit sphere is a distorted version of the usual unit sphere; the distortions will occur along axes corresponding to the eigenvectors of $M$, and the amount of distortion corresponds to the (inverses of the) corresponding eigenvalues. For example when $\dim V = 2$ it is an ellipse and when $\dim V = 3$ it is an ellipsoid.

A Hermitian positive-semidefinite matrix $M$ no longer describes an inner product because it is not necessarily positive-definite, but it still defines a sesquilinear form. It also defines a function $\langle Mv, v \rangle$ which is no longer a norm because it is not necessarily positive-definite; some people call these "pseudonorms," I think. The corresponding unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$ might now be lower-dimensional than the usual unit sphere, depending on how many eigenvalues are equal to zero; for example if $\dim V = 3$ it might be an ellipsoid, or an ellipse, or two points.

share|improve this answer
    
A bit more verbose, but great information! Like the word 'sesquilinear'. :) –  user915 Nov 10 '10 at 21:00
3  
This is a great explanation, because it answers not merely what a positive definite matrix is but why we care. A minor correction: in 3 dimensions, a positive semidefinite matrix $M$ maps the unit sphere to the set $\{Mv:v^Tv=1\}$ which may be an ellipsoid, a filled ellipse, or a line segment, but the set $\{v:v^TMv=1\}$ is either an ellipsoid, an elliptic cylinder, or a pair of parallel planes. –  Rahul Nov 11 '10 at 6:18
    
@Rahul: right. Thanks! –  Qiaochu Yuan Nov 11 '10 at 9:49

Let's consider the set $E$ of all vectors $y=Mx$, where $x\in\mathbb R^n$ belongs to the unit sphere (i.e. $\|x\|=1$). In other words, $E$ is the image of the unit sphere under the linear transformation. If the matrix is non-degenerate, $E$ is an $n$-dimensional ellipsoid. But if we assume additionally that $M$ is symmetric then we can say much more about the structure of $E$. Namely, the directions of the axes of the ellipsoid are pairwise orthogonal and represented by the eigenvectors of the matrix. Moreover, the lengths of the semiaxes are given by the corresponding eigenvalues.

share|improve this answer

Intuitively, it's a matrix that's "like" a single number $\geq 0$.

(Relatedly you can have a positive semidefinite function that's also "like" a number.)

Both matrices and functions in general have many (, many, many) more degrees of freedom than members of $\mathbb R$. But the semidefinite classes of matrices and functions "boil down to" something much simpler.

share|improve this answer
    
Here's my longer explanation. –  isomorphismes Aug 17 '11 at 20:54

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.