The sum of a positive definite matrix and small symmetric matrix Assume $\bf A$ is a (real) positive definite matrix. Let $\varepsilon \neq 0$ be any real (not necessarily positive) number of your choice and $\bf B$ be a fixed (real) symmetric matrix.
Is $\bf A + \varepsilon \bf B$ positive definite? Is it semi-positive-definite? Would anything change if $\bf B$ were not symmetric? Would anything change if $\varepsilon>0$?
The difficulty I'm having is that perhaps $\bf A$ is positive definite but $\bf x^T \bf A \bf x$ can be made arbitrarily small for certain choices of $\bf x$ and so $\bf A + \varepsilon \bf B$ might be negative even though you may choose $\varepsilon$ in advance.
 A: Assume, as in the question, that $A$ is symmetric (real) positive definite and $B$ is symmetric. All matrices are real.  All numbers are real. The set of all matrices on the form
$A+\epsilon B$ is called a matrix pencil, there is a wikipedia aricle on that (too short).
This can be used to define a generalized eigenvalue problem:
$$
   Bv=\lambda A v
$$
solutions $\lambda $ are called generalized eigenvalues and solutions $v$ are called generalized eigenvectors (note that this terms also has other meanings!).  We can define a
generalized Rayleigh ratio as 
$$
R(B,A,x) = \frac{x^T Bx}{x^T Ax}
$$
Since A is positive definite, this will always be defined when $x \not=0$.  Let $\delta$ be the minimum generalized eigenvalue. Then we have, in analogy with the situation for the ordinary Rayleigh quotient (there is a Wikipedia article), that
$$
   R(B,A,x) \ge  \delta \text{~~when $x\not= 0$.}
$$
What does $\delta$ say about $B$? First, note that $\delta=0$ then $B$ is positive semidefinite (and singular).  In that case we can see that
$$
  x^T(A+\epsilon B)x >0
$$   as long as $\epsilon \ge 0$. 
Then we can look at the case $\delta >0$:  Let $\mu_0>0$ be the smallest eigenvalue of $A$.
Vi har
$$
x^T Bx \ge \delta x^T Ax \ge \delta \mu_0 x^Tx
$$
slik at
$\begin{multline}
    \frac{x^T (A+\epsilon B) x}{x^T x} = (x^T Ax+\epsilon x^T Bx )/x^Tx   \\ 
    \ge (x^T Ax +\epsilon \delta \mu_0 x^Tx)/x^T x   \\
    \ge  (\mu_0 x^Tx + \epsilon \delta \mu_0 x^T x)/x^T x    \\
    \ge  \mu_0 (1+\epsilon \delta) > 0
\end{multline} $
and solving the inequality for $\epsilon$ gives 
$$
    \epsilon > -\frac{1}{\delta}
$$
so we can conclude that $A+\epsilon B$ is positive definite if that inequality is fulfilled.
You can solve for the case $\delta <0$ in like manner!
A: Let $\bf A$ be a positive definite real matrix. All matrices, vectors, and numbers are real. Let $\bf x$ be a vector satisfying $|| \bf x||=1$. Then by positive definiteness of $\bf A$ and compactness of the set where $|| \bf x||=1$, there is some $\delta>0$ such that
$$
{\bf x^T  A x} > \delta \qquad (*)
$$
Now choose $\varepsilon$ so that $\varepsilon ({\bf x^T  B x}) \geq -\delta$. Then
$$
{\bf x^T  (A + \varepsilon B ) x} = {\bf x^T  A x} + \varepsilon {\bf x^T  B x} > 
\delta - \delta=0
$$
This leads to $A + \varepsilon  \bf B$ being positive definite on the unit ball and hence positive definite (and hence also semi positive-definite).
Furthermore, nothing changes if $\bf B$ were just an arbitrary finite square matrix or $\varepsilon$ is restricted to be positive (or negative). Nevertheless, the proof fails if $\bf A$ is semi positive-definite (and not positive-definite) because $(*)$ does not hold.
