This is quite an old post, but I choose to answer, because I feel that I offer a completely different understanding of this quote.
I am very surprised to find out my understanding of it is different from others, since when I first read it, I thought to myself: exactly!
To me, this quote is how I feel all throughout studying mathematics. New concepts enter my mind, I learn about their properties and uses, I use them myself, prove theorems with them, yet in the time in between the first sight of the definition and the time when I am fully comfortable with using the concept, there was no aha! moment, when I'd finally understand it.
Take the example of the concept of infinity. You learn about $\lim_{x\to\infty}$, understand Zeno's paradox, $\lim_{x\to c} \frac {f(x)-f(c)}{x-c}$, understand that $\bigcup^{\infty}_{n=1} (0,1-\frac 1n) = (0,1)$, and keep seeing infinity again and again. One has many small epiphanies, but none of them could be considered the moment when one finally understands infinity. And yet there is some kind of road beginning at the first moment of utter confusion as to what infinity actually is and resulting in the feeling of infinity not being all that mysterious at all.
Thus, in this sense, the quote is full of hope. It gives me the reassurance that I don't need to push myself to try to grasp infinity in one evening, there is no piece of information I need to understand in order to say "I got it". Instead, I will gradually get used to its oddness until it becomes a very familiar object.
This process is much better described as getting used to rather than understanding, and thus I understood Neumann's quote in this way and it's been on my mind every time I encounter a new mathematical object.