Proof that the characteristic function of a bounded open set is in $H^{\alpha}$ iff $\alpha < \frac{1}{2}$ Let $u = \chi_{D}$ where $D \subset \mathbb{R}^n$ is an bounded open subset with $C^\infty$ boundary, prove that  $u \in H^{\alpha}$ for $\alpha < \frac{1}{2}$, and also, prove that $u$ is not in $H^{\frac{1}{2}}$.
If $\alpha \in \mathbb{N}$, I can just look at the $L^2$ norms of its distributional derivatives, but here I'm stuck using the norm $\|f|\|_\alpha  =  \int_{\mathbb{R}^n} |\hat{f}|^2 (1+|\zeta|^2)^\alpha$.
I am having a hard time taking the fourier transform of a characteristic function of a general bounded open subset in $\mathbb{R}^n$, so it's hard for me to say anything about $\|f\|_\alpha$ above.
Any suggestions will be appreciated.
 A: As I mentioned in comments, that $\alpha <1/2$ implies the result is already on MSE: To what fractional Sobolev spaces does the step function belong? (Sobolev-Slobodeckij norm of step function). A more general result can be found in this paper. I wrote out the computation slowly in Lemma 6.1 of this preprint.
For the negative result in the case $\alpha = 1/2$ (and therefore $\alpha \ge 1/2$), we lower bound the square of the Gagliardo seminorm, which for indicators $\chi_D$, is the following double integral:
$$[ \chi_D]_{H^{1/2}}^2 = \int_D\int_{D^c}\frac{1}{| x-y|^{1+n}} \, \mathrm{d}x \, \mathrm{d}y\text{.}$$
It is standard (see e.g. the Hitchhiker's guide) that this is equivalent to the squared $L^2(\mathbb R^n)$ norm of $(-\Delta)^{1/2} \chi_D$.
The result is false even without assumptions on the boundary, but it seems that the proof is harder. Other than the above, the only 'technical' tools we use below is a diffeomorphism and some change of variables.
Reduction to local piece with flat boundary
Without loss of generality, $0\in \partial D$. $n=1$ is easy, so suppose $n>1$. As $\partial D\in C^2$ at $0$, there are open neighbourhoods $U,V$ of $0$ and a $C^2$ diffeomorphism $\Phi:U\to V$ with inverse $\Psi$ such that
$$ \Phi(D\cap U)=V\cap \{Y\in\mathbb R^n : Y_n > 0\},
\\ \Phi(D^c\cap U)=V\cap \{X\in\mathbb R^n : X_n \le 0\}.$$
performing a change of variables $x=\Psi(X),\ y=\Psi(Y)$, with $J_\Psi:=|\det\nabla\Psi|$,
\begin{align}
 [ \chi_D]_{H^{1/2}}^2 
&\ge \int_{D\cap U}\int_{D^c\cap U}\frac{1}{| x-y|^{1+n}} \, \mathrm{d}x \, \mathrm{d}y
\\
&=\int_{V\cap (Y_n>0)}\int_{V\cap (X_n\le0)} J_\Psi(X)J_\Psi(Y)\frac{1}{|\Psi(X)-\Psi(Y)|^{1+n}}\,\mathrm{d}X \, \mathrm{d}Y
\\
&=\int_{V\cap (Y_n>0)}\int_{V\cap (X_n\le0)} J_\Psi(X)J_\Psi(Y)\frac{|X-Y|^{1+n}}{|\Psi(X)-\Psi(Y)|^{1+n}} \frac1{|X-Y|^{1+n}} \,\mathrm{d}X \, \mathrm{d}Y
\\
&\ge C \int_{V\cap (Y_n>0)}\int_{V\cap (X_n\le0)} \frac1{|X-Y|^{1+n}} \mathrm{d}X \, \mathrm{d}Y,
\end{align}
where $C = \inf_{X,Y\in V} J_\Psi(X)J_\Psi(Y)\frac{|X-Y|^{1+n}}{|\Psi(X)-\Psi(Y)|^{1+n}} \in(0,\infty)$. As $V$ is an open neighbhourhood of $0$, we can further shrink $V$ to some open box $(-r,r)^n$. At the cost of a multiplicative constant depending on $r$, which we absorb into $C$, we may change variables $(X,Y)=(r\tilde X,r\tilde Y)$ to set $V=(-1,1)^n$. We return to writing $x,y$ for our integration variables. We thus  have, setting $x=(x',x_n),y=(y',y_n)$,
\begin{align}
[\chi_D]_{H^{1/2}}^2
&\ge  C \int_{x'\in[-1,1]^{n-1}}\int_{y'\in[-1,1]^{n-1}}\int_{y_n\in[0,1]}\int_{x_n\in[-1,0]}\frac{\mathrm{d}x_n \,\mathrm{d}y_n \,\mathrm{d}y' \,\mathrm{d}x'}{(|x'-y'|^2+(x_n-y_n)^2)^{(1+n)/2}}
\\
&=C\iint_{x',y'\in[-1,1]^{n-1}}\iint_{x_n,y_n\in[0,1]}\frac{1}{(|x'+y'|^2+(x_n+y_n)^2)^{(1+n)/2}}\,\mathrm{d}x_n \,\mathrm{d}y_n \,\mathrm{d}y' \,\mathrm{d}x'.
\end{align}
Inner two integrals
Define
$$J(r) := \iint_{[0,1]^2} \frac{\,\mathrm{d}a \,\mathrm{d}b}{(r^2 + (a+b)^2)^{n+1}}.$$
Instead of integrating on the square $[0,1]^2$, we lower bound by integrating on the triangle bounded by the axes and the line $a+b=1$. Changing coordinates $u=a+b,v=a-b$ we obtain
\begin{align}
J(r) 
&\ge \frac14 \cdot 2\int_{u=0}^1 \int_{v=0}^u \frac{\,\mathrm{d} v\,\mathrm{d} u}{(r^2+u^2)^{(n+1)/2}} 
\\
&= \frac14\int_{u=0}^1\frac{2u \,\mathrm{d} u}{(r^2+u^2)^{(n+1)/2}} 
\\
&= \frac14\int_{u=0}^1\frac{\,\mathrm{d}(u^2)}{(r^2+u^2)^{(n+1)/2}} 
\\
&= \frac14\left(\frac{-1}{(\frac{n+1}2-1)(r^2+1)^{(n+1)/2-1}} + \frac{1}{(\frac{n+1}2-1)r^{n-1}}  \right)
\end{align}
Divergence
The first term is bounded on $[-1,1]^{2n-2}$, say with integral $\frac{C'}C$, $|C'|<\infty$ and doesn't affect the following calculations;
plugging our lower bound for $J(|x'+y'|)$ and absorbing all constants into $C$, we see
$$[\chi_D]_{H^{1/2}}^2\ge C'+C \iint_{x',y'\in[-1,1]^{n-1}} \frac{dx'dy'}{|x'+y'|^{n-1}}$$
using a similar change of variables as before $u'=x'+y'$, $v'=x'-y'$, and restricting to the region bounded by $|x_i\pm y_i|= 1$ ($i=0,1,\dots,n-1$),
$$[\chi_D]_{H^{1/2}}^2\ge C'+C \int_{v'\in [-1,1]^{n-1}}\,\mathrm{d} v'\int_{u\in [-1,1]^{n-1}}\frac1{|u|^{n-1}}  \,\mathrm{d} u'$$
since $\frac1{|u'|^{n-1}}\notin L^1([-1,1]^{n-1},\,\mathrm{d} u')$, we conclude that $[\chi_D]_{H^{1/2}}^2=\infty$, so $\chi_D\notin H^{1/2}$.
