The answer by agha already explains why the "magic" works. In this answer, I will further elaborate why the method of using integrating factor simplifies the LHS of a differential equation of the form $ y' + yP(x) = Q(x) $ so well.
Consider the linear first-order differential equation
$$ y' + yP(x) = Q(x). \tag{1}\label{eq1} $$
We first find the integrating factor
$$ M(x) = e^{\int P(x)\, dx}. \tag{2}\label{eq2} $$
An Interesting Relationship
Now this integrating factor satisfies an interesting relationship
$$ M'(x) = M(x) P(x). \tag{3}\label{eq3} $$
We can prove this relationship easily by differentiating both sides of \eqref{eq2} as follows:
$$
M'(x)
= \frac{d}{dx} \left( e^{\int P(x)\, dx} \right)
= e^{\int P(x)\, dx} \frac{d}{dx} \left( \int P(x)\, dx \right)
= M(x) P(x).
$$
Note that we use the chain rule to work out the derivative above. This interesting relationship will come useful.
Reducing the LHS to a Single Derivative
Now let us multiply the integrating factor $ M(x) $ on both sides of the differential equation \eqref{eq1}. By doing so, we get
$$
y' M(x) + y P(x) M(x) = Q(x) M(x).
$$
But from \eqref{eq3} we know that $ P(x) M(x) = M'(x) $, so the above equation can be written as
$$
y' M(x) + y M'(x) = Q(x) M(x).
$$
Look what has happened on the left hand side. On the left hand side we have the expansion of $ \frac{d}{dx}(yM(x)) $, that is, by product rule we have $ \frac{d}{dx}(yM(x)) = y' M(x) + y M'(x) $. Therefore the above equation can be written as
$$
\frac{d}{dx}(yM(x)) = Q(x) M(x).
$$
The "magic" has occurred here! Multiplying both sides of the differential equation has led us to an equation that has got a single derivative only on the LHS. So now finding the solution $ y(x) $ is a simple matter of integrating both sides of the previous equation, i.e.,
$$
y M(x) = \int Q(x) M(x) \, dx.
$$
Thus
$$
y = \frac{1}{M(x)} \int Q(x) M(x) \, dx.
$$
Note that the result of indefinite integral on the RHS will contain the constant of integration, which we will denote as $ C $, so the result would look like
$$
y = \frac{1}{M(x)} \int Q(x) M(x) \, dx + \frac{C}{M(x)}.
$$
Illustration
Let us illustrate the method with a very simple differential equation:
$$
y' + \frac{y}{x} = x.
$$
First we note that this is indeed in the form $ y' + yP(x) = Q(x) $ with $ P(x) = 1/x $ and $ Q(x) = x $. We now compute the integrating factor
$$
M(x) = e^{\int P(x) \, dx} = e^{\int \frac{1}{x} \, dx} = e^{\ln x} = x.
$$
Then we multiply the integrating factor on both sides of the differential equation to get
$$
y'x + y = x^2.
$$
Now indeed the LHS can be written down as a single derivative as shown below:
$$
\frac{d}{dx} yx = x^2.
$$
Note that the LHS is the derivative of the product of $ y $ and the integrating factor (exactly as shown in the previous section). Now we integrate both sides,
$$
yx = \frac{x^3}{3} + C.
$$
Finally we divide both sides by the integrating factor (again, exactly as shown in the previous section). The integrating factor for this problem happens to be $ x $, so we divide both sides by $ x $ to get
$$
y = \frac{x^2}{3} + \frac{C}{x}.
$$
We have arrived at the solution $ y(x) $ for the differential equation.
