I'm certainly an advocate of this approach. Here are the criteria I consider important for a side project:
It should be fun.
Here's the unfortunate fact: as beautiful as mathematics is, and as much joy as it can bring you, by the very fact that you care about it so much it will cause you truly epic amounts of stress. And the fact that as a grad student you're surrounded by lots of brilliant people with far more experience than you (heck, your advisor should be one of these people!) doesn't help.
In my mind there are two things a side project can do: it can broaden your horizons mathematically, and it can be something that keeps math fun. Although the former is valuable, it pales in comparison to the latter (at least, in my opinion). Nobody really knows a surefire formula for getting mathematical inspiration, but being happy never hurt. Make your side project something that can remind you why you love math, even if your more serious work is going poorly.
And, more predictably than helping you get inspiration, a fun side project helps you do math every day. Math is like any other skill: practice matters. There are definitely days when one is too stressed out to work on their main project; well, it still works the brain-muscles to fool around with some funny piece of math over yonder.
Incidentally, lest you doubt my seriousness above, my favorite side project came about from a conversation with my advisor which went essentially like this:
"Can I do this with ultrafilters?"
" . . . don't do that with ultrafilters."
"I'mma do that with ultrafilters because reasons."
And it kept me focused and happy throughout my first "serious" project. I am a firm advocate of the value of silliness in mathematics, especially in grad school.