# Are positive definite matrices robust to “small changes”?

Let $A$ be a positive-definite matrix and let $B$ be some other symmetric matrix. Consider the matrix $$C=A+\varepsilon B.$$ for some $\varepsilon>0$. Is it true that for $\varepsilon$ small enough $C$ is also positive definite?

• Is $B$ required to be symmetric? – kimchi lover Jan 2 '18 at 1:02
• I guess, since "positive definite" usually refer to symmetric matrices. I have updated the question. – user_lambda Jan 2 '18 at 1:02
• Eigenvalues of $A+\epsilon B$ depends continuously on $\epsilon$. If they are all positive for $\epsilon=0$ then they are positive for all small $\epsilon$. – A.Γ. Jan 2 '18 at 1:37

Positive definite means that $\langle v, Av \rangle > 0$ for all nonzero vectors $v$; actually it suffices to check this condition for unit vectors. We have

$$\langle v, Cv \rangle = \langle v, Av \rangle + \epsilon \langle v, Bv \rangle.$$

Now, by the compactness of the unit sphere, $\langle v, Av \rangle$ takes on a minimum nonzero value $m$ on unit vectors (the smallest eigenvalue of $A$, although we don't need this), and $| \langle v, Bv \rangle |$ takes on a maximum nonzero value $M$ on unit vectors (the largest eigenvalue of $B$ in absolute value).

So we can take $\epsilon < \frac{m}{M}$, which gives

$$\langle v, Cv \rangle \ge m - \epsilon M > 0$$

for all unit vectors $v$. So $C$ is positive definite as desired.

• Why did you use the spectral radius of $B$ instead of $| \lambda_{\min} (B) |$? – Rodrigo de Azevedo Jan 2 '18 at 10:13
• That would also work. I just needed anything that would clearly give a lower bound. – Qiaochu Yuan Jan 2 '18 at 10:36

If by "positive definite" you mean "strictly positive definite", the answer is "yes". The set of strictly positive definite matrices is an open set in the space of symmetric matrices.

For the following reason. The positive definite matrices are the ones which satisfy a certain finite set of determinental inequalites (the principal minor determinants must all be strictly positive), each one of of which cuts out an open set in the space of matrices. Alternatively, from first principles, let $X$ be the closed unit sphere in vector space, let $Y$ be the symmetric matrices. The function $(v,A)\mapsto \|Av\|$ is continuous, so the set $S=\{(v,A): \|Av\|\le0\}\subset X\times Y$ is closed. Since $X$ is compact, the map $\pi:(v,A)\mapsto A$ is a closed map, so $\pi(S)$ is closed in $Y$, so the complement of $\pi(S)$ is open in $Y$. But that complement is the set of all $A$ for which $\|Av\|>0$ for all $v\in X$, that is to say, the set of all (strictly) positive definite matrices.

• And for positive semi-definite, it's strictly false: $\pmatrix{1 & 0 \\ 0 & 0 } + \epsilon \pmatrix{0 & 0 \\ 0 & -1}$ has eigenvalues $1, -\epsilon$ for every $\epsilon$. – John Hughes Jan 2 '18 at 1:09
• Is "strictly positive definite" different than "positive definite"? I'm unaware of that definition. – user_lambda Jan 2 '18 at 1:14
• @user_lambda: some authors use "positive definite" to mean "positive semidefinite," in the same way that some authors use "positive" to mean "nonnegative." – Qiaochu Yuan Jan 2 '18 at 1:17
• @QiaochuYuan: For what it's worth, I haven't heard of anyone using the word "positive" to mean "non-negative", but what I have seen is the notation of $\mathbb{R}_+$ to mean the non-negative reals. – Mehrdad Jan 2 '18 at 5:42
• @Mehrdad How about Bourbaki? – Daniel Fischer Jan 2 '18 at 15:16

To complement Qiaochu's answer, we have

$$\rm v^\top C \, v = v^\top A \, v + \varepsilon \, v^\top B \, v$$

where $\| \rm v \|_2 = 1$ and $\varepsilon > 0$. Note that

$$\{ \rm v^\top A \, v : \| v \|_2 = 1 \} = [ \lambda_{\min} (\mathrm A), \lambda_{\max} (\mathrm A) ]$$

$$\{ \rm v^\top B \, v : \| v \|_2 = 1 \} = [ \lambda_{\min} (\mathrm B), \lambda_{\max} (\mathrm B) ]$$

and, thus,

$$\{ \rm v^\top C \, v : \| v \|_2 = 1 \} = \left[ \color{blue}{\lambda_{\min} (\mathrm A) + \varepsilon \, \lambda_{\min} (\mathrm B)}, \lambda_{\max} (\mathrm A) + \varepsilon \, \lambda_{\max} (\mathrm B) \right]$$

We know that $\rm A \succ 0$, i.e., $\lambda_{\min} (\mathrm A) > 0$. If $\rm B$ is positive semidefinite, then $\lambda_{\min} (\mathrm A) + \varepsilon \, \lambda_{\min} (\mathrm B) > 0$, i.e., matrix $\rm C$ is positive definite for all values of $\varepsilon > 0$. After all, the conic combination of positive semidefinite matrices is also positive semidefinite. If $\rm B$ is not positive semidefinite, then

$$\lambda_{\min} (\mathrm A) + \varepsilon \, \lambda_{\min} (\mathrm B) = \lambda_{\min} (\mathrm A) - \varepsilon \, |\lambda_{\min} (\mathrm B)| > 0$$

yields the following upper bound on $\varepsilon$

$$\varepsilon < \color{blue}{\frac{\lambda_{\min} (\mathrm A)}{|\lambda_{\min} (\mathrm B)|}}$$