Can we say anything about the first distributional derivatives of $g$, where $g$ is the solution to $-\Delta g =f\in L^p$ given by Riesz potential? If $n\ge3$ and $\omega_n$ is the $n-1$-dimensional Hausdorff measure of the unit sphere in $\mathbb{R}^n$, define:
$$K_n(x):=\frac{1}{(n-2)\omega_n} \frac{1}{|x|^{n-2}}.$$
Then $K_n$ is locally integrable of moderate growth, so it represents by integral pairing a tempered distribution and then it is well defined the convolution operator:
$$K_n*:\mathcal{S}\to C^\infty_\mathcal{M}$$
where $\mathcal{S}$ is the space of Schwartz test functions and $C^\infty_\mathcal{M}$ is the subspace of tempered distributions represented by $C^\infty$-functions of moderate growth with all derivatives of moderate growth.
It can be proved that if $1<p <\frac{n}{2}$ then $K_n*$ extends continuously to an operator:
$$T_p:L^p\to L^{\frac{np}{n-2p}}$$
and:
$$\forall f\in L^p, \forall\varphi\in\mathcal{S}, \int_{\mathbb{R}^n}T_p(f)(x)(-\Delta\varphi(x))\operatorname{d}x = \int_{\mathbb{R}^n}f(x)\varphi(x)\operatorname{d}x,$$
i.e. the distributional laplacian of $T_p(f)$ is $f$.
If $j,k\in\{1,..,n\}$ and if $1<p<+\infty$ it can be proved that the operator:
$$\partial_{jk}^2\circ K_n*:\mathcal{S}\to C^\infty_\mathcal{M}$$
extends continuously to an operator:
$$T_{j,k,p}:L^p\to L^p,$$
and, if $1<p<\frac{n}{2}$, then:
$$\forall f\in L^p, \forall\varphi\in\mathcal{S}, \int_{\mathbb{R}^n}T_p(f)(x)\partial_{jk}^2\varphi(x)\operatorname{d}x= \int_{\mathbb{R}^n}T_{j,k,p}(f)(x)\varphi(x)\operatorname{d}x,$$
i.e. $T_p(f)$ admits second distributional derivatives in $L^p$ and they are given by $(T_{j,k,p}(f))_{j,k\in\{1,...,n\}}$.
What about the first partial derivatives of $T_p(f)$? I.e.:

If $1<p<\frac{n}{2}$ and $f\in L^p$, can we say anything sensible about the first distributional derivatives of $T_p(f)$? For example, can we guarantee that they belongs to any $L^q$?

 A: Yes, we can. Actually, using the fact that $\partial_j K_n$ is positive homogeneous of degree $1-n$, we get that if an estimate of the form
$$\exists C>0, \forall\varphi\in\mathcal{S}, \|\partial_j K_n*\varphi\|_q\le C\|\varphi\|_p,$$
exists at all, then $q=\frac{np}{n-p}.$
In fact, it holds true that for every $p\in[1,n)$ the operator:
$$\partial_j\circ K_n*:\mathcal{S}\to C^\infty_\mathcal{M}$$
is weak-$(p,\frac{np}{n-p})$ with an estimate that can be deduced with an argument in the same spirit to the one used to deduce the corresponding result for $\partial_{jk}\circ K_n*$. Now, we use Marcinkiewicz interpolation to get that $\partial_j\circ K_n*$ is strong-$(p,\frac{np}{n-p})$ for every $p\in(1,n).$ So, for every $1<p<n$ there exists a continuous extension of $\partial_j\circ K_n*$ to an operator:
$$T_{j,p}:L^p\to L^{\frac{np}{n-p}}$$
and we can prove, by a continuity and density argument, that for $1<p<\frac{n}{2}$ it holds true, for every $f\in L^p$, that $T_{j,p}(f)\in L^{\frac{np}{n-p}}$ is the distributional derivative of $T_p(f)\in L^p.$ 
