53
$\begingroup$

What is an intuitive explanation of a positive-semidefinite matrix? Or a simple example which gives more intuition for it rather than the bare definition. Say $x$ is some vector in space and $M$ is some operation on vectors.

The definition is:

A $n$ × $n$ Hermitian matrix M is called positive-semidefinite if

$$x^{*} M x \geq 0$$

for all $x \in \mathbb{C}^n$ (or, all $x \in \mathbb{R}^n$ for the real matrix), where $x^*$ is the conjugate transpose of $x$.

$\endgroup$
118
$\begingroup$

One intuitive definition is as follows. Multiply any vector with a positive semi-definite matrix. The angle between the original vector and the resultant vector will always be less than or equal $\frac{\pi}{2}$. The positive definite matrix tries to keep the vector within a certain half space containing the vector. This is analogous to what a positive number does to a real variable. Multiply it and it only stretches or contracts the number but never reflects it about the origin.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ Why less than $\pi/2$? $\endgroup$ – user10024395 Nov 8 '17 at 6:55
  • $\begingroup$ @user10024395 That's 90 degrees in radians. Any angle beyond that, limiting ourselves to up to 180 degrees, would result in a negative inner product with the original vector, which violates the conditions of the definition. $\endgroup$ – cangrejo Jun 18 '19 at 13:28
  • $\begingroup$ $user10024395, Let us say we have original vector [a b] and then we get a new vector [w1*a w2*b] where both w1 and w2 >0. Due to this the new vector will stay in its original quadrant. I think what user17762 meant was the original quadrant $\endgroup$ – Anant Gupta Jun 21 '19 at 11:12
  • $\begingroup$ @AnantGupta still why quadrant why not bi-quadrant i.e. $\pi$ $\endgroup$ – GENIVI-LEARNER Mar 26 at 21:32
16
$\begingroup$

First I'll tell you how I think about Hermitian positive-definite matrices. A Hermitian positive-definite matrix $M$ defines a sesquilinear inner product $\langle Mv, w \rangle = \langle v, Mw \rangle$, and in fact every inner product on a finite-dimensional inner product space $V$ has this form. In other words it is a way of computing angles between vectors, or a way of projecting vectors onto other vectors; over the real numbers it is the key ingredient to doing Euclidean geometry. An inner product can be recovered from the norm $\langle Mv, v \rangle = \langle v, Mv \rangle$ it induces, and a norm in turn can be recovered from its unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$. This unit sphere is a distorted version of the usual unit sphere; the distortions will occur along axes corresponding to the eigenvectors of $M$, and the amount of distortion corresponds to the (inverses of the) corresponding eigenvalues. For example when $\dim V = 2$ it is an ellipse and when $\dim V = 3$ it is an ellipsoid.

A Hermitian positive-semidefinite matrix $M$ no longer describes an inner product because it is not necessarily positive-definite, but it still defines a sesquilinear form. It also defines a function $\langle Mv, v \rangle$ which is no longer a norm because it is not necessarily positive-definite; some people call these "pseudonorms," I think. The corresponding unit sphere $\{ v : \langle Mv, v \rangle = 1 \}$ might now be lower-dimensional than the usual unit sphere, depending on how many eigenvalues are equal to zero; for example if $\dim V = 3$ it might be an ellipsoid, or an ellipse, or two points.

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ A bit more verbose, but great information! Like the word 'sesquilinear'. :) $\endgroup$ – user915 Nov 10 '10 at 21:00
  • 6
    $\begingroup$ This is a great explanation, because it answers not merely what a positive definite matrix is but why we care. A minor correction: in 3 dimensions, a positive semidefinite matrix $M$ maps the unit sphere to the set $\{Mv:v^Tv=1\}$ which may be an ellipsoid, a filled ellipse, or a line segment, but the set $\{v:v^TMv=1\}$ is either an ellipsoid, an elliptic cylinder, or a pair of parallel planes. $\endgroup$ – user856 Nov 11 '10 at 6:18
  • 2
    $\begingroup$ @Rahul: right. Thanks! $\endgroup$ – Qiaochu Yuan Nov 11 '10 at 9:49
5
$\begingroup$

Let's consider the set $E$ of all vectors $y=Mx$, where $x\in\mathbb R^n$ belongs to the unit sphere (i.e. $\|x\|=1$). In other words, $E$ is the image of the unit sphere under the linear transformation. If the matrix is non-degenerate, $E$ is an $n$-dimensional ellipsoid. But if we assume additionally that $M$ is symmetric then we can say much more about the structure of $E$. Namely, the directions of the axes of the ellipsoid are pairwise orthogonal and represented by the eigenvectors of the matrix. Moreover, the lengths of the semiaxes are given by the corresponding eigenvalues.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

Intuitively, it's a matrix that's "like" a single number $\geq 0$.

(Relatedly you can have a positive semidefinite function that's also "like" a number.)

Both matrices and functions in general have many (, many, many) more degrees of freedom than members of $\mathbb R$. But the semidefinite classes of matrices and functions "boil down to" something much simpler.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Here's my longer explanation. $\endgroup$ – isomorphismes Aug 17 '11 at 20:54
  • 2
    $\begingroup$ The link is dead as of Sep 2017. Might be here. $\endgroup$ – akraf Sep 27 '17 at 14:22
1
$\begingroup$

Positive-definite matrices are matrices that are congruent to the identity matrix, i.e. that can be written as $P^HP$ for invertible $P$ (for some reason, a lot of authors define congruence as $N=P^TMP$, but here we go by the Hermitian definition $N=P^HMP$).

One reason this is useful is that if two forms $M$ and $N$ are congruent, their corresponding "generalised unitary groups" $\{A^HMA=M\}$ and $\{B^HNB=N\}$ are isomorphic (via conjugation by $P$). So positive-definite matrices (as well as negative-definite matrices, because $-I$ is preserved by the unitary group as well) define a dot product whose geometry is isomorphic to Euclidean geometry.

Similarly, a positive semidefinite matrix defines a geometry that Euclidean geometry is homeomorphic to -- to put it in slightly imprecisely, such a geometry has all the symmetries of Euclidean geometry, and perhaps then some.

See a fuller treatment here.

| cite | improve this answer | |
$\endgroup$
-1
$\begingroup$

Let us take an example of a vector V = i + 2j which in matrix notation is

enter image description here

It will be represented as enter image description here

  • POSITIVE DEFINITE MATRIX

A positive definite matrix is

enter image description here

How do we know it is positive definite matrix

enter image description here

The altered vector is

enter image description here

enter image description here

  • POSITIVE SEMI-DEFINITE MATRIX

enter image description here

The altered vector is [0 , 0]

  • NEGATIVE MATRIX

enter image description here

The altered vector in this case is

enter image description here

We can see the quadrant has changed

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ why do we care if the quadrant has changed or not? Also how is the graph for semi-definite looks like? Can you please elaborate more on what do you mean by altered vector $\endgroup$ – GENIVI-LEARNER Feb 9 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy