In a text book I am following, there is a definition of positive definite matrices, which I did not see before:

$x^TAx \geq \alpha x^Tx$

where $\alpha$ is a positive scalar and $x \in \mathbb{R^n}$.

The usual definition I know is for every non-zero vector $x \in \mathbb{R^n}$, it is $x^TAx > 0$.

These definitions must be equivalent. It is trivial to show that the first definition implies $x^TAx > 0$. But I failed to see a way to show the other way around of the equivalence. I thought of applying SVD to the matrix $A$ and tried to find a way to show that $x^TAx$ is always lower bounded by a $\alpha x^Tx$ with $\alpha$ being positive, using singular values and vectors of $A$ but couldn't move forward. What is the correct way of approach here?

(The other question in the board assumes symmetric matrices. This one does not).


marked as duplicate by Martin R, Hans Lundmark, Paul Frost, Jyrki Lahtonen, Christopher Nov 26 '18 at 14:15

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ I think we can prove that, assuming that $A$ is PD, there exists $a$ such that $A-aI$ is PSD, that is because the eigenvalues of $A-aI$ will be bigger than those of $A$ minus $a$ (this needs to be proven) and since they were strictly positive in the first place then you can find $a$ such that they still are positive. $\endgroup$ – P. Quinton Nov 26 '18 at 7:25

Let $f(x)=x^TAx$. Furthermore let $f(x)>0$ for all non-zero $x$.

Let $C:=\{x \in \mathbb R^n: ||x||_2=1\}$. Then $C$ is compact and $f$ is continuous on $C$. Thus there is $x_0 \in C$ such that

$f(x_0) \le f(x)$ for all $x \in C$.

Now put $ \alpha =f(x_0)$. Then $\alpha >0$.

It is your turn to show that $x^TAx \geq \alpha x^Tx$ for all $x \in \mathbb R^n$.

Hint: for $t \in \mathbb R^n$ and $t \in \mathbb R$ we have $f(tx)=t^2f(x).$

  • $\begingroup$ Thank you for the answer. I think we have the following: $C$ is compact and $f(x)$ is a continuous mapping, which implies $f(C)$ is also compact, which additionally means $f(C)$ is closed and bounded. This provides us with a $x_0 \in C$ such that $\alpha = f(x_0) = \inf f(C)$ and $f(x_0) \in f(C)$. For any $x \in \mathbb{R}^n$, we have $\left(\dfrac{x}{||x||_2} \right)^T A \left(\dfrac{x}{||x||_2} \right) \geq \alpha$. Seeing that $x^Tx = ||x||_2^2$ we multiply both sides with $x^Tx$ and then we are good to go: $x^T Ax \geq \alpha x^T x$. $\endgroup$ – Ufuk Can Bicici Nov 26 '18 at 14:44
  • 1
    $\begingroup$ Well done ! In $\left(\dfrac{x}{||x||_2} \right)^T A \left(\dfrac{x}{||x||_2} \right) \geq \alpha$ you should mention that $x \ne 0$. $\endgroup$ – Fred Nov 26 '18 at 14:57

The following is valid only if the matrix $A$ doesn't have complex values, which is at least true if it is also symmetric.

Since $A\succ 0$, we can use it's diagonalisation as $A=U\Sigma U^{\dagger}$ such that $U U^\dagger = U^\dagger U = I$ and $\Sigma$ is the diagonal matrix containing the eigenvalues of $A$, then for any $a>0$, $A-aI = U(\Sigma - aI) U^\dagger$ and so the eigenvalues of $A-aI$ are those of $A$ minus $a$.

Being positive definite means that the eigenvalues are strictly positive and so we can find $0<a\leq\min_{i} \lambda_i$ where $\lbrace\lambda_i\rbrace_{i\in[1{:}n]}$ are the eigen values such that the eigen values of $A-aI$ are positive which is $A\succ aI$ and hence $x^\dagger A x \geq a x^\dagger x$

  • $\begingroup$ $A$ can be nonsymmetric, so its eigenvalues do not have to be real; then how can we say that positive definite means the eigenvalues are strictly positive? $\endgroup$ – Ufuk Can Bicici Nov 26 '18 at 9:11
  • $\begingroup$ Well, I guess this is one of the properties of the positive definite matrix en.wikipedia.org/wiki/… Also if an eigenvalue $\lambda$ is complex, then if $v$ is the associated eigenvector, then $v^\dagger A v = \lambda v^\dagger v$ which is a complex value and cannot be compared to $av^\dagger v$ so the problem is not solvable. $\endgroup$ – P. Quinton Nov 26 '18 at 9:17
  • $\begingroup$ Interestingly the question here: math.stackexchange.com/questions/83134/… shows complex eigenvalues of a positive definite matrix. I think being symmetric is implicitly assumed in the text then. $\endgroup$ – Ufuk Can Bicici Nov 26 '18 at 9:19
  • 1
    $\begingroup$ Then I guess you would need to compare the real values in the question, otherwise it doesn't make sense. And yes you are right, in the wikipedia page they assume symetric. I think Fred's way is the way to go, but you should add real values everywhere. $\endgroup$ – P. Quinton Nov 26 '18 at 9:21

Not the answer you're looking for? Browse other questions tagged or ask your own question.