Cubes may be open, closed or neither. They are of the form $\prod _{i}\left \langle a_{i},b_{i} \right \rangle$ where the symbol $\left \langle \right \rangle$ refers to $\textit {any}$ type of interval.
You can always enclose a cube $Q$ of any of these types in an open cube $Q'$ that is as "close" to $Q$ as you want.
In fact there is a more general statement that includes your case:
Fix $Q$ and define $S=\left \{ x\in R^{d}:d(x,Q)<\epsilon \right \}$ where $d(x,Q)=\inf_{y\in Q}\left \{ d(x,y) \right \}$. If we can show that $f(x)=d(x,Q)$ is continuous, this will prove that $S=d^{-1}((0,\epsilon))$ is open which is what we want.
So, pick a point $z\in Q$ and note that by the triangle inequality, we have
$d(x,z)\leq d(x,y)+d(y,z)$. This inequality is true for all $z\in Q$ so in fact
$$\tag1d(x,Q)\leq d(x,y)+d(y,Q)$$ Likewise, we have
$d(y,z)\leq d(x,y)+d(x,z)$ and therefore $$\tag2d(y,Q)\leq d(x,y)+d(x,Q)$$
Combining $(1)$ and $(2)$, we have now
$$\tag3\left | d(x,Q)-d(y,Q) \right |\leq d(x,y)$$
and we see now that $(3)$ proves continuity as soon as we set $\delta =\epsilon$.