Here's where understanding the physical content of a problem is just as important as understanding it mathematically. The Van Der Waals interaction can be thought of as the net electrostatic attraction or repulsion between 2 atoms caused solely by the minor but constantly relative charge of each atom due to the rotating of each nucleii's "electron cloud". For example, say we have 2 iodine atoms. Since iodine has a large and relatively diffuse electron cloud (why is complicated,don't worry about it) .as the cloud rotates around it's nuclei, it's not symmetrical. As a result, the nuclei is partially exposed when the cloud "bulges" to one side and this causes a momentary net positive charge on this side,while the "bulge" side has a net negative charge. Now imagine these kinds of transitory charge imbalances happen throughout all the iodine atoms in a given sample of iodine gas.Then at any given moment, the "average" of these charges may be either net positive or net negative. Positive charges will attract negative ones, resulting in a net attractive force. Like charges will repel, so likewise a net repulsive force results in this case. The force of the net charge effect at any given moment is the Van der Waals interaction. It turns out to have immense importance in chemistry.For example, Van Der Walls interaction is one of the reasons why gases in the real world don't obey the ideal gas law.
In any event, like any net force, the VDW interaction has both a potential and kinetic energy function. Notice the potential energy function is an inverse function of the radius r of the nuclei to the electron cloud (the greater r is,the further away from the nuclei the electrons are and therefore the Van Der Waals interaction potential energy diminishes). That's why we need the negative derivative to obtain the force of the interaction. Remember from basic physics force is a vector and direction matters!
The important thing to realize in solving this problem is that the potential energy function can be expressed in terms of either $x$ or $r$ depending on how you choose to express it from the relation $x = r - R_0$. In either case, note that $U$ is a composition of functions. $U$ is a function of $x$ which in turn is a function of $r$.
Remember-we got rules around here for that kind of thing! (HINT!)