51
$\begingroup$

What is an intuitive explanation of a positive-semidefinite matrix? Or a simple example which gives more intuition for it rather than the bare definition. Say $x$ is some vector in space and $M$ is some operation on vectors.

The definition is:

A $n$ × $n$ Hermitian matrix M is called positive-semidefinite if

$$x^{*} M x \geq 0$$

for all $x \in \mathbb{C}^n$ (or, all $x \in \mathbb{R}^n$ for the real matrix), where $x^*$ is the conjugate transpose of $x$.

$\endgroup$
  • $\begingroup$ en.wikipedia.org/wiki/Positive-definite_matrix have you seen this $\endgroup$ – anonymous Nov 10 '10 at 20:20
  • 1
    $\begingroup$ What's not intuitive that requires explanation? $\endgroup$ – Qiaochu Yuan Nov 10 '10 at 20:20
  • 6
    $\begingroup$ Here I mean intuitive in a vector-geometry sense. The definition is very straightforward, but I'm trying to find out the motivation for its definition, etc. $\endgroup$ – user915 Nov 10 '10 at 20:37
111
$\begingroup$

One intuitive definition is as follows. Multiply any vector with a positive semi-definite matrix. The angle between the original vector and the resultant vector will always be less than or equal $\frac{\pi}{2}$. The positive definite matrix tries to keep the vector within a certain half space containing the vector. This is analogous to what a positive number does to a real variable. Multiply it and it only stretches or contracts the number but never reflects it about the origin.

$\endgroup$
  • $\begingroup$ Why less than $\pi/2$? $\endgroup$ – user10024395 Nov 8 '17 at 6:55
16
$\begingroup$

First I'll tell you how I think about Hermitian positive-definite matrices. A Hermitian positive-definite matrix $M$ defines a sesquilinear inner product $\langle Mv, w \rangle = \langle v, Mw \rangle$, and in fact every inner product on a finite-dimensional inner product space $V$ has this form. In other words it is a way of computing angles between vectors, or a way of projecting vectors onto other vectors; over the real numbers it is the key ingredient to doing Euclidean geometry. An inner product can be recovered from the norm $\langle Mv, v \rangle = \langle v, Mv \rangle$ it induces, and a norm in turn can be recovered from its unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$. This unit sphere is a distorted version of the usual unit sphere; the distortions will occur along axes corresponding to the eigenvectors of $M$, and the amount of distortion corresponds to the (inverses of the) corresponding eigenvalues. For example when $\dim V = 2$ it is an ellipse and when $\dim V = 3$ it is an ellipsoid.

A Hermitian positive-semidefinite matrix $M$ no longer describes an inner product because it is not necessarily positive-definite, but it still defines a sesquilinear form. It also defines a function $\langle Mv, v \rangle$ which is no longer a norm because it is not necessarily positive-definite; some people call these "pseudonorms," I think. The corresponding unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$ might now be lower-dimensional than the usual unit sphere, depending on how many eigenvalues are equal to zero; for example if $\dim V = 3$ it might be an ellipsoid, or an ellipse, or two points.

$\endgroup$
  • 2
    $\begingroup$ A bit more verbose, but great information! Like the word 'sesquilinear'. :) $\endgroup$ – user915 Nov 10 '10 at 21:00
  • 5
    $\begingroup$ This is a great explanation, because it answers not merely what a positive definite matrix is but why we care. A minor correction: in 3 dimensions, a positive semidefinite matrix $M$ maps the unit sphere to the set $\{Mv:v^Tv=1\}$ which may be an ellipsoid, a filled ellipse, or a line segment, but the set $\{v:v^TMv=1\}$ is either an ellipsoid, an elliptic cylinder, or a pair of parallel planes. $\endgroup$ – Rahul Nov 11 '10 at 6:18
  • 1
    $\begingroup$ @Rahul: right. Thanks! $\endgroup$ – Qiaochu Yuan Nov 11 '10 at 9:49
5
$\begingroup$

Let's consider the set $E$ of all vectors $y=Mx$, where $x\in\mathbb R^n$ belongs to the unit sphere (i.e. $\|x\|=1$). In other words, $E$ is the image of the unit sphere under the linear transformation. If the matrix is non-degenerate, $E$ is an $n$-dimensional ellipsoid. But if we assume additionally that $M$ is symmetric then we can say much more about the structure of $E$. Namely, the directions of the axes of the ellipsoid are pairwise orthogonal and represented by the eigenvectors of the matrix. Moreover, the lengths of the semiaxes are given by the corresponding eigenvalues.

$\endgroup$
3
$\begingroup$

Intuitively, it's a matrix that's "like" a single number $\geq 0$.

(Relatedly you can have a positive semidefinite function that's also "like" a number.)

Both matrices and functions in general have many (, many, many) more degrees of freedom than members of $\mathbb R$. But the semidefinite classes of matrices and functions "boil down to" something much simpler.

$\endgroup$
  • $\begingroup$ Here's my longer explanation. $\endgroup$ – isomorphismes Aug 17 '11 at 20:54
  • 1
    $\begingroup$ The link is dead as of Sep 2017. Might be here. $\endgroup$ – akraf Sep 27 '17 at 14:22
1
$\begingroup$

Positive-definite matrices are matrices that are congruent to the identity matrix, i.e. that can be written as $P^HP$ for invertible $P$ (for some reason, a lot of authors define congruence as $N=P^TMP$, but here we go by the Hermitian definition $N=P^HMP$).

One reason this is useful is that if two forms $M$ and $N$ are congruent, their corresponding "generalised unitary groups" $\{A^HMA=M\}$ and $\{B^HNB=N\}$ are isomorphic (via conjugation by $P$). So positive-definite matrices (as well as negative-definite matrices, because $-I$ is preserved by the unitary group as well) define a dot product whose geometry is isomorphic to Euclidean geometry.

Similarly, a positive semidefinite matrix defines a geometry that Euclidean geometry is homeomorphic to -- to put it in slightly imprecisely, such a geometry has all the symmetries of Euclidean geometry, and perhaps then some.

See a fuller treatment here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy