Yes, the boundary points are precisely the points which, in the second set, satisfy $y=0$.
There are two things to prove in order to see this.
First, suppose that $p$ has a neighborhood $U$ with a homeomorphism $f$ taking $U$ to the second set, and suppose that $f(p)=(x,y)$ with $y>0$. Then $p$ has a different neighborhood homeomorphic to the first set, namely
$$U' = f^{-1} \biggl(\{(x,y) \in \mathbb{R}^2 \,\bigm|\, x^2+y^2<1, y>0\} \biggr)
$$
and its not hard to show by direct construction that this set is homeomorphic to
$$\{(x,y) \in \mathbb{R}^2 \,\bigm|\, x^2 + y^2 < 1\}
$$
Second, it is not possible for $p$ to have neighborhoods $U,V$, a homeomorphism $f$ from $U$ to the first set, and a homeomorphism $g$ from $V$ to the second set, such that $g(p)=(x,0)$. Arguing by contradiction, suppose that $p$ did have two such neighborhoods. We may assume that $f(p)=(0,0)$ and $g(p)=(0,0)$ (by rechoosing the neighborhoods and homeomorphisms, if necessary). Then $f$ has a neighborhood basis
$$U = U_1 \supset U_2 \supset \cdots
$$
and homeomorphisms $f_i$ from $U_i$ to the first set such that $f_i(p)=(0,0)$: let $U_i$ equal $f^{-1}$ of the ball of radius $1/i$ around $(0,0)$, and let $f_i$ equal the composition of $f | U_i$ with a scalar multiplication. Similarly, $p$ has a neighborhood basis
$$V = V_1 \supset V_2 \supset \cdots
$$
and homeomorphisms $g_j$ from $V_j$ to the second set such that $g_j(p)=(0,0)$. Choose $j$ so that $V_j \subset U_1$, then choose $i$ such that $U_i \subset V_j$, consider the inclusion maps
$$U_i \subset V_j \subset U_1
$$
consider a point $q \in U_i$, and consider the induced homormorphisms
$$\pi_1(U_i,q) \to \pi_1(V_j,q) \to \pi_1(U_1,q)
$$
Since $V_j$ is simply connected, this composition of homomorphisms is the zero homorphism. However, it is obviously true that
$$\mathbb{Z} \approx \pi_1(U_i,q) \to \pi_1(U_1,q) \approx \mathbb{Z}
$$
is an isomorphism, which is a contradiction.
