Speaking as someone at a U.S. institution, my experience suggests that three or four research letters is standard, and fine. Less than three research letters is probably not enough, and if there are many more than four, the file starts to become a bit long to read. However, cardinal's comment above is also important: there is no point diluting your application by adding letters which are not as strong as the existing letters. Of course, you don't know what your letters will say, but as a proxy: once you have three sensibly chosen letter writers, there is no point adding a fourth or fifth if they are not as familiar with you or your work (since the chance that their letters will be weaker, and so dilute your application, is greater than the chance of their letters going the other way).
Of course, your advisor knows all this. But as well as suggesting that you get four letters, hopefully your advisor has suggested four names to write those letters! If so, your advisor probably knows the four suggested letter writers, and is reasonably confident about the letters they will write, in which case I don't think there is any need to go against your advisor's suggestion.