Determine probability that one set is closer to a random point than another set? Suppose we have two sets that never intersect on a interval.
How do we rigorously determine the probability that one set is closer to a random point than another set. (Closest being the set intersects with the random point). How do we make this rigorous? Can we create a new measure?
By intuition I expect, for example
1) The irrationals would have a probability of 1 compared to the rationals. A random point will always be "on" the irrationals, while the rationals are infinitesimally close.  
2) A set countable and Dense in R would have a probability of 1 compared to a set countable and non-dense in R.
3) A set countable and dense in R cannot have a probability compared to another set countable and dense in R. Both sets can be as close to the random point as possible.
 A: I'm going to work inside $[0,1]$ instead of $\mathbb{R}$ to "normalize" things. This isn't essential, but makes more sense in a context where we want the maximum "size" to be $1$ (given that we're talking in terms of probability).

Per Steven Stadnicki's answer, distance is close to but not quite what we want here. However, as far as I can tell the only twist you want to add is that we should distinguish between "arbitrarily close to" and "exactly" - e.g. an irrational should be closer to $[0,1]\setminus\mathbb{Q}$ than to $[0,1]\cap\mathbb{Q}$.
At this point we can introduce a simple notion of preference. For $A,B$ disjoint sets, say that $x$ prefers $A$ to $B$ iff we have $$\exists a\in A\forall b\in B[d(x,a)<d(x,b)].$$ 
Some quick observations: 


*

*$\sqrt{2}$ prefers $[0,1]\setminus\mathbb{Q}$ to $[0,1]\cap\mathbb{Q}$ as desired - take $a=\sqrt{2}$. More generally, if $x\in A$ and $x\not\in B$ then $x$ prefers $A$ to $B$.

*If $A,B$ are each dense and $x\not\in A\cup B$ then $x$ doesn't prefer $A$ to $B$ or $B$ to $A$.
Now given two disjoint sets $A$ and $B$, we get two new sets $A_{\triangleright B}$ and $B_{\triangleright A}$ given by $$A_{\triangleright B}=\{x: x\mbox{ prefers $A$ to $B$}\}\quad\mbox{and}\quad B_{\triangleright A}=\{x: x\mbox{ prefers $B$ to $A$}\}.$$ We'll always have $A\subseteq A_{\triangleright B}$, $B\subseteq B_{\triangleright A}$, and $A_{\triangleright B}\cap B_{\triangleright A}=\emptyset$. Of course there may be points in neither set.

Incidentally, it's easy to actually compute $A_{\triangleright B}$: it's just $$A\cup(cl(A)\setminus cl(B))\cup \{x: d(x,A)<d(x,B)\},$$ where $d(y,U)=\inf\{d(y,u): u\in U\}$. That is: everything in $A$ already, plus everything "infinitesimally close" to $A$ which isn't "infinitesimally close" to (or in) $B$, plus everything strictly closer to $A$ than to $B$ in the usual sense.

It seems, then, that the probability you're interested in is just the ratio of the measures of these sets: $$Prob(A\triangleright B)={m(A_{\triangleright B})\over m(B_{\triangleright A})}$$ if that ratio is defined. 
This agrees with each of the examples you've given: e.g. if $A,B$ are disjoint, dense, and countable, then we have $A_{\triangleright B}=A$ and $B_{\triangleright A}=B$, each of which have measure zero, yielding an undefined probability. But note that all of this still boils down to a measure calculation - namely, we need to assign numbers to $A_{\triangleright B}$ and $B_{\triangleright A}$ somehow in order to get a ratio to care about. 
A: You can certainly define such a quantity, but it won't do what you want.  For instance, the notion of distance between a point and a set can be defined as $d(p, A) = \inf_{q\in A} d(p,q)$; you can then say that the 'relative closeness index' of $A$ over $B$ (on, say, the unit interval) is the measure of the set $S\subseteq[0,1] = \{s: d(s,A)\lt d(s,B)\}$.  I'd bet (but don't hold me to this) that this set is even guaranteed to be measurable and that therefore the value you're after is well-defined.
But the problem is that it won't solve the problems you're after, because $d(p,A)$ can't distinguish between $A$ and $\mathrm{cl}(A)$ at all; since you're taking an infimum, adding in limit points doesn't affect the result at all. In particular, $d(p,\mathbb{Q}\cap[0,1]) = d(p, (\mathbb{R}-\mathbb{Q})\cap[0,1]) = 0$ for all $p\in[0,1]$, since there are both rationals and irrationals arbitrarily close to any point.
