The problem to be solved was:
Find the values of $p$ so that the equation $2\cos^2 x - (p+3)\cos x + 2(p-1) = 0$ has real roots.
This is not the same as a problem like finding $p$ such that the equation
$ 2 y^2 - (p+3)y + 2(p-1) = 0 $ has real roots, because we don't just have
a polynomial in some variable $y$; the equation also involves $\cos x$.
When we ask for real roots we are asking for real numbers that can be plugged in for $x$ so as to make the equation $2\cos^2 x - (p+3)\cos x + 2(p-1) = 0$ true.
Now since $x$ must be real, we know $\cos x$ also will be real
(the cosine of any real number is a real number).
More specifically, $\cos x$ will be a real number belonging to the closed interval $[-1,1].$
But it can be any number in that interval; if we let $x$ run from $0$ to $\pi$ then $\cos x$ will hit every number in $[-1,1].$
The thing is, if we have to start worrying about the "interval $[-1,1]$" part right away, it's hard to see how to solve the problem.
So let's just worry about the "real number" part at first.
We want to know that there is a real number that you can put in place of $\cos x$
in the equation that makes the equation true.
Or to put it slightly differently, let's define the variable $y$ by the
relationship $y = \cos x,$ and now we want to know that there is a real number $y$ that satisfies
$$ 2 y^2 - (p+3)y + 2(p-1) = 0 . $$
You already know how to do that: you must ensure that $b^2 - 4ac \geq 0.$
In this particular problem,
$$b^2 - 4ac = (p + 3)^2 - 16(p-1) = p^2 - 10p + 25 = (p - 5)^2.$$
Now observe that $(p - 5)^2 \geq 0$, and therefore
$b^2 - 4ac \geq 0,$ for any real number $p.$
That is, for any real number $p,$ there is at least one real number, usually two, that could be the value of $y$ in the true equation $ 2 y^2 - (p+3)y + 2(p-1) = 0. $
But now we have to remember that $\cos x$ cannot be just any real number.
It has to be in the interval $[-1,1].$
So instead of just being concerned about whether a real number exists that makes
$ 2 y^2 - (p+3)y + 2(p-1) = 0 $ true when we plug this number in for $y,$
we need to think about which real number we might have to plug in for $y,$
and make sure that that number is in the interval $[-1,1].$
The use of the variable $p$ makes it a little difficult to guess a factorization of the polynomial $2 y^2 - (p+3)y + 2(p-1),$ so let's just apply the standard formula to find solutions of any solvable quadratic equation in $y$:
$$ y = \frac{-b \pm \sqrt{b^2 - 4ac} }{2a}. $$
Putting $a = 2,$ $b = -(p+3),$ $c = 2(p-1),$ we have
$$ y = \frac{(p+3) \pm \sqrt{(p+3)^2 - 16(p-1)} }{4}
= \frac{(p+3) \pm (p - 5)}{4}. $$
If we choose $+$ for $\pm$ we get $y = (p - 1)/2$ and if we choose $-$ we get
$y = 2.$
Obviously $y = 2$ is no good because $2$ is not in the interval $[-1,1]$
so it cannot be a value of $\cos x.$
So our only hope is that the solution is $y = (p - 1)/2.$
In order for this to be a value of $\cos x$ (so that it solves the problem that was originally given) it must be true that
$-1 \leq (p - 1)/2 \leq 1.$
The rest of the solution is just algebra to simplify these inequalities.
The simplified version is $-1 \leq p \leq 3.$
Now it may seem that the book's solution did not actually start by requiring that
$b^2 - 4ac \geq 0$ and asking what $p$ made that inequality true.
You might ask what allows them to skip that step.
The reason they can do it is because the test for $b^2 - 4ac \geq 0$ is itself
just a consequence of the formula
$$ y = \frac{-b \pm \sqrt{b^2 - 4ac} }{2a}. $$
If the expression under the $\sqrt{\phantom{0}}$ symbol is negative then it does not have a real square root and the formula does not give you any solutions.
So that expression has to be non-negative, that is, $b^2 - 4ac \geq 0.$
And if all you care about is whether there is any solution,
not what particular number the solution is, $b^2 - 4ac \geq 0$ is all you need to know, because then there will be a real square root and you'll combine it with other real numbers to get a real number (or two) in the end.
(I'm assuming that you're working in real numbers only and not complex numbers.
If you allow complex numbers then there is a non-real square root and we have to do some extra work to show that the final result of calculating $y$ is not real.)
Since we actually need to know more about $y$ than just that it's a real number,
we need to finish evaluating the formula. In the book's solution they go directly to that step, but while simplifying $\sqrt{(p+3)^2 - 16(p-1)}$ to $\pm(p-5)$
they had to figure out that $(p+3)^2 - 16(p-1) = (p - 5)^2,$
which implies that $b^2 - 4ac \geq 0.$
