I think I have a proof that the claim is true. It seems to follow from a theorem of Khinchin in $1924$. This theorem asserts the following. Consider an arbitrary function $\psi:\mathbb{N} \to \mathbb{R}_{\geq 0}$ such that $\{q \psi(q)\}_{q \in \mathbb{N}}$ is decreasing, and let $\mathcal{K}$ denote the set of real numbers $\alpha$ for which $$\left|\alpha-\frac{p}{q}\right|<\frac{\psi(q)}{q}$$
has infinitely many solutions $p/q \neq \alpha$. Then if $\sum_{q \geq 1} \psi(q)=+\infty$, $\mathcal{K}$ has full measure. Note that it follows that $\mathbb{R} \setminus \mathcal{K}$ has measure zero, and this complement set consists precisely of those $\alpha$ such that $\left|\alpha-\frac{p}{q}\right|<\frac{\psi(q)}{q}$ has at most finitely many solutions $p/q \neq \alpha$.
I have taken the formulation of Khinchin's theorem from this paper by Dimitris Koukoulopoulos and James Maynard (not quite verbatim). In their paper, everything was done in the interval $[0,1]$, but replacing $[0,1]$ with $\mathbb{R}$ is immaterial—I've reworded things accordingly.
Now, going back to the original problem, our set $A$ can be equivalently written as $$A=\bigcup_{N \geq 1} A_N$$ where $$A_N=\left\{\alpha \in \mathbb{R} : \left|\alpha-\frac{p}{q}\right| \geq \frac{1}{Nq^2} \ \text{for all rational} \ p/q \neq \alpha \right\}$$
It thus suffices to prove each $A_N$ has measure zero, by the subadditivity of measure. We note that $A_N \subset A'_N$, where we define $A'_N$ similar to $A_N$, but with "for all rational $p/q$" replaced by "for all but possibly finitely many rational $p/q$". It suffices to prove $A'_N$ has measure zero. Here, taking $\psi(q)=\frac{1}{qN}$, we have by Khinchin's theorem that $A'_N$ is exactly $\mathbb{R} \setminus \mathcal{K}$, where $\mathcal{K}$ is as defined earlier. Moreover $\sum_{q \geq 1} \frac{1}{qN}=+\infty$. It follows that $A'_N$ has measure zero. This proves the result.
Note we also have a simple corollary. For fixed $\lambda>2$, say a real number $\alpha$ is $\lambda$-badly-approximable if $\left|\alpha-\frac{p}{q}\right| > \frac{C}{q^\lambda}$ for all rational $p/q \neq \alpha$ and some positive constant $C=C(\lambda, \alpha)$. Let $A_{\lambda}$ denote the set of $\lambda$-badly-approximable numbers. Then $A_{\lambda}$ has full measure. In other words, almost all real numbers are $\lambda$-badly-approximable. This is in stark contrast to the case $\lambda=2$, which we've shown has zero measure. The reason this holds is that the full statement of Khinchin's theorem states not only that if $\sum_{q \geq 1} \psi(q)=+\infty$, $\mathcal{K}$ has full measure, but it also states that if $\sum_{q \geq 1} \psi(q)<+\infty$, $\mathcal{K}$ has zero measure. Thus we can mimick the argument earlier, this time using $\psi(q)=\frac{1}{Nq^{\lambda-1}}$, and using that $\mathbb{R} \setminus \mathcal{K} = A_{\lambda}$ and $\sum_{q} \frac{1}{Nq^{\lambda-1}} < + \infty$.
I also suspect that Khinchin's theorem can be used to prove the fact that almost all real numbers have irrationality measure equal to $2$ (unless of course Khinchin used this fact in the original proof!)
