The * point in @M.sina's answer is really useful. However, I had a hard time following his sketch and failed to find it elsewhere, so I decided to write a more detailed proof. Hope this isn't off topic for this question.
*. Let $B$ be a closed subset and $A$ a compact subset of a topological group $G$ such that $A\cap B=\emptyset.$ Then there exists a neighborhood $U$ of $e$ such that:
$$1. AU\cap BU=\emptyset\\2. UA\cap UB=\emptyset$$
Proof: Since $B$ is closed, $G\setminus B$ is open and for every $a_i \in G \setminus B$, there exists a symmetric neighborhood $V_{i1}$ of $e$ such that $a_iV_{i1}^2\cap B=\emptyset$ and a symmetric neighborhood $V_{i2}$ of $e$ such that $V_{i2}^2a_i\cap B=\emptyset$, take $V_i=V_{i1}\cap V_{i2}$. $V_i$ is also open, symmetric, and has the disjoint property.
Since $\bigcup_{\{i|a_i \in G\setminus B\}} (a_iV_i\cap V_ia_i)$ covers $G\setminus B \supseteq A$ and $A$ is compact, there are some $a_1,\dots,a_n$ and neighbourhoods $V_1,\ldots,V_n$ of $e$ such that $\bigcup_{i=1}^n (a_iV_i\cap V_ia_i)\supseteq A$. Then for any $a\in (a_iV_i\cap V_ia_i)$ we have $aV_i \subseteq a_iV_i^2$ and $V_ia \subseteq V_i^2a_i$, so $V=\bigcap_{i=1}^n V_i$ satisfies $AV\cap B=\emptyset$ and $VA\cap B=\emptyset$.
Then find some symmetric neighborhood $U$ of $e$ such that $U^2 \subseteq V$. We have
$$\emptyset=AV\cap B\supseteq AU^2\cap B$$
Suppose that $AU\cap BU\neq \emptyset $, then there exist $au_1=bu_2\Rightarrow au_1u_2 ^{-1}=b$, contradicting $AU^2\cap B= \emptyset $, so $AU\cap BU=\emptyset$. Similarly, $\emptyset=VA\cap B\supseteq U^2A\cap B$, so $UA\cap UB=\emptyset$.