There are a few related mathematical notions that correspond to the existence of "holes". The one that is usually introduced first is the notion of non-simply connected space. A simply-connected space is a space that is path-connected and whose fundamental group is trivial. See this Wikipedia article for more information. So if the fundamental group is not trivial , like in the case of an annulus, where it is isomorphic to $(\mathbb{Z},+)$, then this implies that there is some sort of 1-dimensional hole.
However, one cannot use the fundamental group to completely characterize the "holes" of a topological space, as the fundamental group is defined based on using paths between points, so intuitively, it can only detect 1-dimensional behavior. Thus one can go further by studying higher homotopy, where instead of closed paths (maps from circles), we use $n$-spheres.