How can one relate inverse of a differential operator to an integral operator? Informally speaking, the integral operator can be regarded as the inverse of some differential operator. In some very special case, finding the inverse of the differential operator is equivalent to finding the Green's function. For example if the differential operator $A$ is defined as $$Au:=au''+bu'+cu,$$ then in some BVP, one may have 
$$A^{-1}f(x)=\int_{0}^1g(x,y)f(y)dy$$

If one knows that
  $A^{-1}f=(I-\partial_{xx}^2)^{-1}\partial_xf$,
  how can one turn it into the form
  above, i.e, write it as an integral?

By solving an ODE, one may find $(I-\partial_{xx}^2)^{-1}$. However, when it comes to $(I-\partial_{xx}^2)^{-1}\partial_x$, things become complicated. 

Is there any rule of thumb for dealing
  with this kind of problems?

 A: For linear, constant coefficient problems, you can use the Fourier transform. 
Take $P$ to be an arbitrary constant coefficient linear partial differential operator on $\mathbb{R}^n$. Associated to $P$ is its polynomial symbol $p(\xi)$. Formally speaking, one can treat a constant coefficient linear partial differential operator as a polynomial in the usual partial derivations $\partial_{x^i}$. We say that a (complex-coefficient) polynomial $p(\xi)$ over $\mathbb{R}^n$, where $\xi = (\xi_1, \xi_2, \ldots, \xi_n)$, is the symbol of $P$ if we formally insert $p(i\nabla)$ we will recover the operator $P$. 
Some examples, for the partial derivative $\partial_{x^1}$, you get $-i\xi_1$. For the Laplacian $\sum_{i = 1}^n \partial_{x^i}^2$, you get $- \sum_{i = 1}^n \xi_i^2$. 
The reason we care about the symbol is because they are nice under the Fourier transform. Namely, the Fourier transform of $Pf(x)$ (say $f$ is Schwartz class) is precisely $p(\xi)\hat{f}(\xi)$. 
So for if $f$ and $u$ are Schwartz functions such that
$$ Pf = u $$
we get
$$ p\hat{f} = \hat{u}$$
by taking the Fourier transform of both side, and since now $p$ is a polynomial, we can divide through by
$$ \hat{f} = \frac{1}{p} \hat{u} $$
Now, if $\frac{1}{p}$ were actually a Schwartz function, then we can take the inverse Fourier transform on both sides and use that the Fourier transform interchanges products and convolutions, to arrive at
$$ f = \check{(p^{-1})} * u $$
and your convolution kernel $g(x,y)$ is nothing more than just what is given by the inverse Fourier transform
$$ g(x,y) = \check{(p^{-1})}(x-y) $$
Unfortunately, since $p$ is a polynomial, $1/p$ can never be in Schwartz class. There are two problems. The first is generic: since $p$ is polynomial, $1/p$ can have at most polynomial decay near $\infty$, so can never be "rapidly decaying" as required by Schwartz functions. This, however, can usually be amended by considering enlargement of the domain of Fourier transform to include tempered distributions. (In some cases you can even get by with using the $L^2$ theory for Fourier transform.) The second, slightly less generic problem, is that of zeros of $p$. When $p$ has a zero, $1/p$ has a singularity, and so is not smooth, and hence not in Schwartz class. The question about how to deal with this problem has driven much of mid twentieth century harmonic analysis, and has led to the study of singular integrals. 

A discussion of singular integrals is way beyond the scope here. And for distribution theory I suggest the book of Friedlander and Joshi. In your particular case, the operator $(I - \partial^2_{xx})$ has symbol $1 + \xi^2$, so you don't have the second type of problem described above. The only issue is that the symbol for $(I-\partial^2_{xx})^{-1}\partial_x$ is $(-i\xi)/(1 + \xi^2)$ which is not even $L^2$. So you will have to use the theory of distributions to get the convolution kernel. 

Lastly, the Fourier transform method rather strongly depends on your domain being the whole real line (or some $\mathbb{R}^n$). For boundary value problems it doesn't work so well. In the case of the interval, you can use some Fourier series methods (by considering things on a circle), but it tends to interact strangely with prescription of boundary values. 
A: This kind of problems is dealt with the theory of pseudodifferential operators. A long story...
You can find a nice discussion in a recentish MO thread.
