7
$\begingroup$

I'm trying to get some intuition behind the meaning of a positive semidefinite matrix, which I learned a long time ago in undergrad but clearly didn't internalize properly.

As I understand, a symmetric matrix $M \in \textbf{R}^{n~\times~n}$ is positive semidefinite iff $z^TMz \ge 0$, $\forall z \in \textbf{R}^n$. Note that I'd like to use this particular definition, not a more general one that involves complex numbers. As such, $z^TMz \in \textbf{R}$.

This definition makes sense to me, and this question clarified it further, but then I was reading Boyd's textbook and became confused by an unrelated definition explained in $\S$3.1.4, which implies that the Hessian matrix $\textbf{H}$ of function $f$ is positive semidefinite if $\textbf{H} \succcurlyeq 0$, where the $"\succcurlyeq"$ symbol refers to a componentwise inequality between matrices.

Thus, can a positive semidefinite matrix contain negative entries?

EDIT: This question turned out to be silly, but if you have this question and am as rusty with linear algebra as I am, this post might be useful.

$\endgroup$
  • 4
    $\begingroup$ A quick and dirty way to see that there can be some negative entries is to think of adding a large positive multiple of the identity to a completely arbitrary symmetric matrix. This shifts all the eigenvalues up as much as you want without breaking the symmetry. $\endgroup$ – Ian Jul 14 '17 at 18:10
  • 1
    $\begingroup$ Also notice that your definition of "positive semidefinite" is wrong. In fact, there is no n×n matrix M such that x^T M x > 0 for all vectors x. Indeed, trivially x^T M x = 0 if x is the n-dimensional zero vector. The correct definition is that M is positive semidefinite iff x^T M x >= 0 for all x. If, in addition, equality only happens for the zero vector, we say that M is positive definite. Then x^T M x > 0 for all non-zero vectors x. $\endgroup$ – Andreas Rejbrand Jul 14 '17 at 19:58
  • $\begingroup$ @AndreasRejbrand It was a typo - I fixed it. $\endgroup$ – mchen.ja Jul 14 '17 at 20:09
11
$\begingroup$

The symbol $\succeq$ actually does not denote componentwise inequality for matrices in that book. If $A$ and $B$ are symmetric matrices then $A \succeq B$ means $A-B$ is positive semidefinite. Perhaps confusingly, if $x$ and $y$ are vectors rather than matrices, then in that case $x \succeq y$ does mean that each component of $x$ is greater than or equal to the corresponding component of $y$.

(As Lord Shark the Unknown mentioned, a positive semidefinite matrix can have some negative entries.)


Here are a few more details. On p. 43, Boyd and Vandenberghe introduce the notation $x \preceq_K y$ (where $K$ is a proper cone) to mean that $y - x \in K$. If $K$ is the nonnegative orthant then $x \preceq_K y$ means that $y - x$ is in the nonnegative orthant, or in other words that $y_i \geq x_i$ for all $i$. If $K$ is the positive semidefinite cone then $A \preceq_K B$ means that $B - A$ belongs to the positive semidefinite cone, or in other words that $B - A$ is positive semidefinite.

The book goes on to state

The nonstrict and strict partial orderings associated with the nonnegative orthant arise so frequently that we drop the subscript $\mathbb R^n_+$; it is understood when the symbol $\preceq$ or $\prec$ appears between vectors.

Regarding the case where $K$ is the positive semidefinite cone, the book comments

Here, too, the partial ordering arises so frequently that we drop the subscript.

$\endgroup$
  • $\begingroup$ ahhh. whoops 😅 $\endgroup$ – mchen.ja Jul 14 '17 at 16:03
  • 1
    $\begingroup$ @mchen.ja I made an edit to answer that question. $\endgroup$ – littleO Jul 14 '17 at 16:16
  • $\begingroup$ And as alephzero mentioned, it can have many negative entries. $\endgroup$ – Andreas Rejbrand Jul 14 '17 at 20:05
12
$\begingroup$

Can a positive semidefinite matrix contain negative elements? If by elements, you mean entries, then yes, $$\pmatrix{2&-1\\-1&2}$$ is positive definite.

$\endgroup$
  • 4
    $\begingroup$ Any diagonally dominant symmetric matrix is positive definite. So there are positive definite matrices of any size, where every off diagonal term is negative. $\endgroup$ – alephzero Jul 14 '17 at 19:22
  • 1
    $\begingroup$ @alephzero: That is not right. You also need the right sign on the diagonal entries (of course). $\endgroup$ – Andreas Rejbrand Jul 14 '17 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.