The $f^{-1}(y)$ is locally constant where $y$ ranges through regular values

Let $f: M \longrightarrow N$ be a smooth map between two manifolds of the same dimension, with M compact, and a regular value $y \in N$. Then the number of points in $f^{-1}(y)$ is locally constant as a function of $y$, where $y$ ranges through regular values.

I read the proof in Milnor's book "Topology from the differentiable viewpoint" but if I'm not wrong this proof concerns regular values with preimage. But I don't know how to prove it in the case where $y$ is not the preimage of some $x \in M$. In this case $f^{-1}(y)$ should be locally $0$ in a neighborhood of $y$, right?

Notice that the source $M$ is compact, so the image is closed. Therefore, the complement of the image in $N$ is open.