Here is an alternative approach similar to my solution of this problem: Closed form of integral over fractional part $\int_0^1 \left\{\frac{1}{2}\left(x+\frac{1}{x}\right)\right\}\,dx$. The closed form expression of the integral is traced back to the asymptotic behaviour of $g(n) = \sum_{k=1}^n \log(k)^2$.
1. Calculation
Letting $x=1/y$, and splitting the resulting integration range into equidistant pieces from $k$ to $k+1$ ($k=1,2,3,...$) gives
$$i:=\int_{0}^1 \frac{\log(x)}{x} (\{\frac{1}{x}\}-\frac{1}{2})\,dy=-\int_{1}^\infty \frac{\log(y)}{y} (\{y\}-\frac{1}{2})\,dy=\sum_{k=1}^\infty a(k)$$
Where
$$a_k = -\int_0^1 \frac{\left(\xi -\frac{1}{2}\right) \log (k+\xi )}{k+\xi } \, d\xi
\\
=-\frac{1}{2} \log ^2(k+1)+\frac{1}{4} \left(\log ^2(k+1)-\log ^2(k)\right)+\frac{1}{2} \left((k+1) \log ^2(k+1)-k \log ^2(k)\right)-((k+1) \log (k+1)-k \log (k))+1$$
Forming the partial sum of $a_k$ most of the terms telescope witht the result
$$i_s(n) := \sum_{k=1}^n a_k = i_{s1}(n) + i_{s2}(n)$$
$$i_{s1}(n) = -(-n+(n+1) \log (n+1)-\frac{1}{4} \log ^2(n+1)-\frac{1}{2} (n+1) \log ^2(n+1))$$
$$i_{s2}(n) = -\frac{1}{2}g(n+1)$$
With
$$g(n) = \sum_{k=1}^{n}(\log(k))^2$$
In order to find the asymptotic behaviour of $g(n)$, we notice first that
$$\nu(n,x) := \sum_{k=1}^n k^x =H(n,-x)$$
where $H$ is the generalized harmonic number, is a generating function for our finite sum.
Its asymptotic expression is provided by Mathematica:
$$\nu_a(n,x) = \left(\frac{-x^3+3 x^2-2 x}{720 n^3}+\frac{n}{x+1}+\frac{x}{12 n}+\frac{1}{2}\right) n^x+\zeta (-x)$$
Hence we have
$$g_{a}(n) = \frac{\partial ^2 \nu_a(n,x)}{\partial x^2}| x\to 0 \\
= \frac{1}{120 n^3}+2 \left(-\frac{1}{360 n^3}-n+\frac{1}{12 n}\right) \log (n)+2 n+\left(n+\frac{1}{2}\right) \log ^2(n)+\gamma _1+\frac{\gamma ^2}{2}-\frac{\pi ^2}{24}-\frac{1}{2} (\log (2)+\log (\pi ))^2$$
The asymptotics of $i_{s1}(n)$ is easily calculated
$$i_{s1}(n)\to \frac{5}{12 n^2}-\frac{5 \log (n)}{12 n^2}+n+\frac{1}{2} n \log ^2(n)+\frac{3 \log ^2(n)}{4}-n \log (n)+\frac{\log (n)}{n}-1$$
Finally,
$$i_{s} = \lim_{n\to\infty} (i_{s1}(n)+i_{s2}(n))
\\=-\frac{\gamma_1}{2}-\frac{\gamma^2}{4}+\frac{\pi^2}{48}-1+\frac{1}{4} (\log(2 \pi ))^2= 0.0031782279542924256051 $$
in agreement with the previously obtained result of Olivier Oloa.
2. Discussion
In the OP it was stated that Mathematica returns a strange result when the integral is calculated directly (I assume NIntegrate was used). I confirm that. So the representation as an infinite sum is better suited for numerical purposes.
The following graph shows how the partial sums approoach the limiting value with increasing number of terms

The changing sign of the partial sums indicates that a false value $0$ can be understood on the basis of missing accuracy.