Question related to a proof about the multiplicity of some eigenvalues

I have a question related to Lemma 4.2 from this pdf (which is, btw quite a nice exposition of Hoffman Singleton work on the classifications of Moore graphs of diameter 2 and 3.)

We are given a $n \times n$ real symmetric matrix $A$ with eigenvalues $d,\lambda_1,\lambda_2$ where $d$ has multiplicity 1. The key part of Lemma 4.2 computes the multiplicity of $\lambda_1$ and $\lambda_2$ when these two values are irrational.

The part I don't understand is when it tries to deduce that if $\lambda_1$ and $\lambda_2$ are irrational then they have both equal multiplicity $\frac{n-1}{2}.$ Somehow I don't understand the part when it says that if $(x-\lambda_1)$ divides $p(x)$ then $(x-\lambda_1)(x-\lambda_2)$ also divisides $p(x).$

How does that follows? Is anyone able to clarify the proof of the Lemma to me?

-

As far as I can see, $A$ isn't an arbitrary real matrix, but rather the adjacency matrix of some graph, and thus contains only values $0$ and $1$. It follows that the characteristic polynomial $p$ is in $\mathbb{Z}[x]$.
The trick is then to realize that if the multiplicities of $\lambda_1,\lambda_2$ do not agree, then $p(x)$ cannot be a polynomial over $\mathbb{Z}[x]$. Their argument seems a bit strange to me, though - I don't believe that it's generally true that $(x-\lambda_1)(x-\lambda_2)$ is a polynomial over $\mathbb{Z}$, but rather over $\mathbb{Q}$. You'll thus need to multiply with some constant to get rid of all the fractions, but at that point, why would the result necessarily be a factor of $p(x) \in \mathbb{Z}[x]$?
I believe a more elementary argument should suffice, though. Simply watch what happens if you expand $(x-\lambda_1)^{n_1}(x-\lambda_2)^{n_2}$ where $\lambda_1=a+\sqrt{b}$,$\lambda_2=a-\sqrt{b}$. If $n_1 \neq n_2$, those squares won't cancel, and thus you won't get a polynomial over $\mathbb{Z}$.
Or you could argue (less elementary) that $\lambda_1,\lambda_2$ are algebraically indistinguishable over $\mathbb{Z}$, so if one of them is a zero of some polynomial, the other must be too.