Given two vertices $x$ and $y$, $N(x, y) = N(x) \cap N(y)$ is the common neighbourhood of those two vertices where the size would be denoted as $|N(x) \cap N(y)|$.
What is "common" neighborhood for a single vertex?
It seems a bit superfluous to use the term "common neighbourhood" when referring to a single vertex since the neighbours that a vertex has in common with itself is all of its neighbours.
$$
N(x, x) = N(x) \cap N(x) = N(x) \tag{Idempotent law}
$$
I think the authors of the paper are primarily concerned with comparing distinct vertices in partition $V$. This is covered in section "Candidate selection" which describes why selecting candidates in non-decreasing order of common neighbourhood size might reduce the number of non-maximal subsets that the algorithm has to generate. So in Figure 5 for graph $G_4$, they are sorting based on $|N(v_i, v_{j})|$, which in this example results in the algorithm not picking candidate vertex $v_1$ first.