Proving that a symmetric matrix is positive definite iff all eigenvalues are positive

This has essentially been asked before here but I guess I need 50 reputation to comment. Also, here I have some questions of my own.

My Proof outline: (forward direction/Necessary direction): Call the symmetric matrix $A$. Write the quadratic form for $A$ as $x^{t}Ax$, where superscript $t$ denotes transpose. $A$ p.d. (positive definite) implies $x^{t}Ax >0 \ \forall x\neq 0$.

if $v$ is an eigenvector of A, then $v^t Av \ =v^t \lambda v \ =\lambda \ >0$ where $\lambda$ is the eigenvalue associated with $v$. $\therefore$ all eigenvalues are positive.

Any hints for the reverse direction? Perhaps I need to write $A$ as $PDP^{-1}$ where D is a diagonal matrix of the eigenvalues of A and the columns of P are eigenvectors?

Also, a more general question but one that is probably important, is that, since the statement does not assume that A is real (in addition to symmetric), does the possibility of complex entries introduce any complications? Do I need to show that the eigenvalues are real?

Thanks.

• the result is only true for real matrices. For complex matrices they need to by hermitian. As for your proof, notice that you may select $P$ such that it is orthogonal / unitary. – user251257 Sep 13 '15 at 22:32
• I don't see how orthogonal helps me. I need to show that for any $x \neq 0$ $x^t PDP^{-1}x = x^t PDP^{t}x >0$ If P is orthogonal. I just don't see how I can simplify the statement enough to be able to say that it is greater than 0 (because I don't see how I can characterize/generalize the $PDP^{-1}$ into something I can work with. – majmun Sep 13 '15 at 22:49
• $x^t P D P^t x = (P^t x)^t D (P^t x)$. Now, do you see that a diagonal matrix with positive diagonal components is positive definite? – user251257 Sep 13 '15 at 22:51
• One way I can see it (that I should have seen before), is that all of D's leading principle minors are positive so it is positive definite (and therefore $(P^t x)^t D(P^t x) >0$ implying A is positive definite. That should take care of the proof. I feel like D being positive definite should be obvious without the use of a theorem, though, in which case I am still missing something (sorry). When I think about $x^t PDP^t x=(P^t x)tD(P^t x)$ I believe I can conclude that its diagonal is all positive elements, if that means anything. – majmun Sep 13 '15 at 23:07
• Let $e_1,\dotsc, e_n$ denote the standard basis. Then, $(\sum_{i=1}^n x_i e_i) ^t D (\sum_{j=1}^n x_j e_j) = \sum_{i,j=1}^n x_i x_j e_i^t D e_j = ... = \sum_{i=1}^n x_i^2 D_{i,i} > 0.$ – user251257 Sep 13 '15 at 23:19

This is a (sketch of a) proof when the symmetric matrix $A$ is real. Let $u_1, \ldots, u_n$ be the linearly independent eigenvectors which correspond to the positive eigenvalues $\lambda_1, \ldots, \lambda_n$ of the real symmetric matrix $A$. Also, let $z = c_1 u_1 + \cdots + c_nu_n$ be a random $n \times 1$ real vector with $z\neq \vec 0$. Thus, we have:

$z^TAz \begin{array}[t]{l}= (c_1 u_1^T + \cdots +c_n u_n^T) P D P^{-1}(c_1 u_1 + \cdots +c_n u_n)\\\\ = \begin{pmatrix} c_1 \|u_1\|^2_2 & c_2 \|u_2\|^2_2& \cdots & c_n \|u_n\|^2_2 \end{pmatrix}\cdot \begin{pmatrix} \lambda_1 & 0 & \cdots & 0\\ 0 & \lambda_2 & \cdots & 0\\ \vdots & \vdots & \ddots & \vdots\\ 0 & 0 &\cdots & \lambda_n \end{pmatrix}\cdot \begin{pmatrix} c_1\|u_1\|^2_2 \\ c_2\|u_2\|^2_2 \\ \vdots \\ c_n\|u_n\|^2_2 \end{pmatrix}\\ =\lambda_1 c_1^2 +\cdots + \lambda_n c_n^2, \end{array}$

which is clearly positive due to the positive $\lambda_i$'s and it also holds $\|u_i\|_2^2 =u_i^T\cdot u_i= 1$.

I think you can fill in the details.

Just to clear things out..

Because matrix $A$ is a real symmetric one, it can be written in the form $$A = P \cdot D \cdot P^{-1} = P \cdot D \cdot P^T,$$ where the columns of $P$ contain the right hand eigenvectors of matrix $A$ and $P^{-1} (= P^T$) contain the left hand eigenvectors as its rows. Thus, if $u_i$ 's are the right hand eigenvectors, then $u_i^T$'s are the left hand eigenvectors of $A$. Then, it holds: $$u_i^T \cdot u_j = \begin{cases} 1, & i = j\\ 0,& i\neq j \end{cases}$$

• you need that $u_1, \dotsc, u_n$ are orthogonal – user251257 Sep 13 '15 at 23:16
• @user251257 Yes, you are right! Since $A$ is a real, symmetric matrix, we can assume that it can be written in the form $P D P^{-1},$ where $P$ is an orthogonal matrix. – thanasissdr Sep 13 '15 at 23:18
• One (perhaps very dumb) question: you had $(c_1 u_1 + \cdots +c_n u_n)$. Then in the next stop you only have the column vector of $(c_1 + \cdots +c_n)$. What happened to the $u_n$s? Did you somehow rearrange terms to arrive at the norm squared on the left hand side? If so, what property allows you to rearrange terms as such? Wouldn't doing so violate non-commutativity of matrix multiplication? – majmun Sep 13 '15 at 23:18
• @thanasissdr no worries. Also, so (geometrically) they are orthogonal to each other, since their dot product is 0. Also, why are we discussing left and right eigenvectors? Shouldn't the eigenvectors in this problem be all right or all left eigenvectors? – majmun Sep 13 '15 at 23:52
• @user265678 Answer edited. I should have been a little more specific in our case. I think it is a little bit clearer now.. – thanasissdr Sep 14 '15 at 0:35

If $A$ is symmetric and has positive eigenvalues, then, by the spectral theorem for symmetric matrices, there is an orthogonal matrix $Q$ such that $A = Q^\top \Lambda Q$, with $\Lambda = \text{diag}(\lambda_1,\dots,\lambda_n)$. If $x$ is any nonzero vector, then $y := Qx \ne 0$ and
$$x^\top A x = x^\top (Q^\top \Lambda Q) x = (x^\top Q^\top) \Lambda (Q x) = y^\top \Lambda y = \sum_{i=1}^n \lambda_i y_i^2 > 0$$
since $y$ is nonzero and $A$ has positive eigenvalues.