$\def\diamond{\diamondsuit}$
Modal logic is concerned with the logic of so-called "modal operators", often "necessarily true" and "possibly true", which are symbolized with $\square$ and $\diamond$ respectively.
The idea is that while it is true that George Bush was the 43rd president of the United States, it is not necessarily true, because one can easily imagine a slightly different world in which Al Gore was president instead—it was the closest of close elections. Considering the statements:
$$
B = \text{The 43rd president was Bush}\\
G = \text{The 43rd president was Gore}
$$
We can say unequivocally that $B$ is true and $G$ is false. But we could also say that $B$ is not necessarily true; that is $\lnot\square B$, or equivalently that it is possible that $B$ could have been false: $\diamond\lnot B$. And we could say that $G$ is possibly true; that is $\diamond G$, or equivalently that $G$ is not necessarily false; that is $\lnot\square\lnot G$.
On the other hand if we consider the statement:
$$ P = \text{$131$ is a prime number} $$
we can't reasonably think of a possible world in which $P$ is false, so we might say that $P$ is "necessarily" true; that is $\square P$, or equivalently that it is not possibly false: $\lnot\diamond\lnot P$.
Now any sophomore philosophy major could argue for hours about whether it was really possible for Gore to have been elected president in 2000 or whether it is necessarily true that 131 is a prime number. So we sidestep such arguments about what "necessarily true" and "possible world" actually mean and consider a generalization. In the generalization, we consider some set $W$ of world-states, some set of "possible worlds". And we say that some of these worlds are "accessible" from others: for example, we might like to say that before the 2000 election, it was possible that Gore would have been the 43rd president, and after the election it was no longer possible. Our set $W$ might include a state $E$ before the election, and a state $B$ where Bush had been elected, and a state $G$ where Gore had been elected, and then we might say that $G$ is accessible from $E$ but not from $B$; we find ourselves now in $B$. Once we have chosen this "accessibility" relation, we take “$\diamond S$ is true in world $W$” to mean “There exists some world $W'$, accessible from $W$, in which $S$ is true”. And we take “$\square S$ is true in world $W$” to mean “For every world $W'$, accessible from $W$, it is the case that $S$ is true”.
The idea is not to try to construct a "set of all possible worlds", which I agree with you might be philosophically and mathematically incoherent. All we are doing is selecting some reasonable set of worlds to consider, for the purpose of understanding behaviors of modal operators such as $\square$ and $\diamond$.
It develops that this is an interesting thing to do! Certain intuitively reasonable axioms for the modal operators correspond naturally to certain simple conditions on the "accessible" relation. For example, consider the very reasonable axiom that $\square p\to p$. This says that if $p$ is necessarily true in some world $W$, then it in fact true in $W$. This axiom holds exactly if the accessibility relation is reflexive; that is, if $W$ is accessible from itself for each possible world $W$ under consideration.
One similarly likes to consider modal operators such as "could become true in the future" and "was false in the past", where the accessibility relation is usually understood to represent the evolution of time; "is known to be true", "is provable", and "is consistent with ZF"; "is permitted", and so forth. Each of these has different properties, and is formalized by different axioms, and gives rise to a different sort of accessibility relations on the possible worlds. The intuitions about the meanings of the modal operators help inform the intuitions about the accessibility relations, and vice versa.
But again, the "possible worlds" don't represent the entirety of the universe. They are only abstractions that have certain simplified relations that allow us to capture interesting features of logical operators. In this way they are not at all different from other mathematical abstractions.
I hope this clears something up for you.