If $\mathbf A$ and $\mathbf B$ are Hermitian, when does $\det(\mathbf A-\lambda\mathbf B)=0$ have only real roots? Let $\mathbf A, \mathbf B$ be Hermitian matrices of the same size. What is the characterization of $\mathbf A, \mathbf B$ such that $p(\lambda)=\det(\mathbf A-\lambda\mathbf B)=0$ has only real roots? 
If $\mathbf B$ is positive definite (I corrected this), it is easy to see $p(\lambda)$ has only real roots.
 A: If $\det(A - \lambda B) = 0$, there exists some vector $v$ such that $(A-\lambda B)v = 0$. Multiply through by $\bar{v}^T$, you get
$$ \bar{v}^TAv = \lambda \bar{v}^TB v $$
Since $A$ is Hermitian, 
$$ \bar{v}^TAv = v^T A^T \bar{v} = v^T \bar{A} \bar{v} \in \mathbb{R}$$
so if $\bar{v}^T B v \neq 0$, the corresponding $\lambda$ is real. 

BTW, your statement that 

if $B$ is positive semidefinite, it is easy to see $p(\lambda)$ has only real roots

is false. Let 
$$ A = B = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix} $$
then $\det( A - \lambda B) = 0$ for any complex number $\lambda$. 

It is also not enough that $B$ be non-degenerate. Let 
$$ A = \begin{pmatrix} 0 & 1 \\ 1 & 0\end{pmatrix} \qquad B = \begin{pmatrix} -1 & 0 \\ 0 & 1\end{pmatrix} $$
The polynomial $p(\lambda) = -\lambda^2 - 1$ and has no real roots. This happens because the "eigenvector"s in this problem are the vectors $(1,1)^T$ and $(1,-1)^T$

Of course, a sufficient condition would be that $B$ is positive or negative definite. But this is far from necessary. A slightly more general result is Theorem 10.1 in the paper of Lancaster and Rodman

Theorem Let $A$ and $B$ be Hermitian. Suppose there exists a pair of real numbers $\alpha,\beta$ such that $C:= \alpha A + \beta B$ is positive semi-definite with $\ker C \subset \ker A \cap \ker B$, then $A$ and $B$ are simultaneously diagonalisable. 

Note that this rules out the second example (for which there cannot be $\alpha,\beta$ to make $C$ positive semi-definite), but this is not sufficient to deal with the first example: if $\ker A\cap \ker B$ is non-empty, the polynomial $p(\lambda) \equiv 0$. 
In general the problem is quite complicated, and the problem you want to ask is not the one that you asked. You should not be asking about roots of $p(\lambda)$ (since if $B$ is not non-degenerate, $p(\lambda)$ may have degree smaller than the full dimension), but you should be asking about whether $A$ and $B$ can be simultaneously diagonalized. For that I suggest you consult the linked paper and the references given in Section 10. 
A: As an addendum of sorts to Willie's answer:

Theorem (Stewart 1979): If $\mathbf A$ and $\mathbf B$ are both Hermitian and
  $\mathbf x^\ast(\mathbf A+i\mathbf B)\mathbf x$ is nonzero for all nonzero $\mathbf x$,
  then there exists a real number $t$ such that the matrix
  $\mathbf A\sin\,t+\mathbf B\cos\,t$ is positive definite.

In particular, there is an algorithm (based on this paper by Crawford and Moon) for finding a scalar $t$ such that the pencil $\mathbf F-\mu\mathbf G$ is a symmetric definite pencil, where $\mathbf F=\mathbf A\cos\,t-\mathbf B\sin\,t$ and $\mathbf G=\mathbf A\sin\,t+\mathbf B\cos\,t$. If $\lambda$ satisfies $\det(\mathbf A-\lambda\mathbf B)=0$ ($\lambda$ is an eigenvalue of the pencil $\mathbf A-\lambda\mathbf B$), then
$$\lambda=\frac{\sin\,t+\mu\cos\,t}{\cos\,t-\mu\sin\,t}$$
and the eigenvectors of both pencils are the same.
The Crawford-Moon algorithm was subsequently improved by Guo, Higham, and Tisseur in this paper.
Additionally, Frobenius has already shown (in 1910) that any real square matrix can be expressed as a product of two symmetric matrices, one of the two being nonsingular; thus any unsymmetric eigenvalue problem $\mathbf A\mathbf x=\lambda\mathbf x$ (which as you already know might exhibit complex eigenvalues) can be recast as a generalized symmetric eigenproblem $\mathbf B\mathbf y=\lambda\mathbf C\mathbf y$, where $\mathbf A=\mathbf C^{-1}\mathbf B$ and $\mathbf B$ and $\mathbf C$ are symmetric. (See this more recent paper for proofs.)
