Notice that in the equality you want to prove:
$$
\int_{-\infty}^{\infty} u(x,\color{red}{t})\,dx = \int_{-\infty}^{\infty}g\, dx,
$$
the left term is a function of the temporal variable while the right term is a real constant assuming that $u(x,0) = g\in L^1(\mathbb{R})$ and is compactly supported as well. If for the moment we assume $u$ is smooth, take the time derivative on the left:
$$
\frac{d}{d\color{red}{t}}\int_{-\infty}^{\infty} u(x,t)\,dx = \int_{-\infty}^{\infty} \frac{\partial}{\partial \color{red}{t}}u(x,t)\,dx,\tag{1}
$$
where we performed the differentiation under the integral sign for $u$ is smooth. The right side of above is:
$$
\int_{-\infty}^{\infty} \frac{\partial}{\partial t}u(x,t)\,dx = -\int_{-\infty}^{\infty} \frac{\partial}{\partial x} F\big(u(x,t)\big)\,dx =- F\big(u(x,t)\big)\Big\vert^{\infty}_{-\infty}=0,
$$
because of $u$ is compactly supported and $F(0) = 0$. Hence
$$
\int_{-\infty}^{\infty} u(x,t)\,dx = \int_{-\infty}^{\infty}u(x,0)\, dx = \int_{-\infty}^{\infty}g\, dx.
$$
This gives you an idea of why this is called conservation law.
Back to the integral solution $u$ of the conservation law (the definition is somewhere earlier in that conservation law chapter of Evans):
$$
\int^{\infty}_0 \int^{\infty}_{-\infty} \Big(u v_t + F(u) v_x\Big)\,dxdt + \int^{\infty}_{-\infty} gv\,dx \big|_{t=0}= 0,\tag{2}
$$
for $v\in C^{\infty}_c(\mathbb{R}\times [0,\infty))$. Now $u$ only lies in $L^{\infty}\cap C_c$ ($u$ may not be differentiable anymore, thinking all those blows up in time, and shock waves in space!), the trick in (1) is not applicable anymore, here the way to prove this is to choose proper test function $v$.
Think $u(x,\tau)$ for any $\tau>0$, and consider the problem when the time starts at $\tau$:
$$
\left\{ \begin{aligned}
u_t + F(u)_x &= 0 &\text{ in } \mathbb{R} \times (\tau,\infty), \\
u&=u(x,\tau) &\text{ at } \mathbb{R} \times \left\{t=\tau\right\} ,
\end{aligned} \right.\tag{$\star$}
$$
The weak solution to $(\star)$ coincides with the original IVP if we assume the solution is unique (but unfortunately this is not true for the integral solution, that's why we solve for an entropy solution by artificially adding a diffusion perturbation, which is the so-called vanishing viscosity method). Then we can see $u$ satisfies:
$$
\int^{\infty}_{\tau} \int^{\infty}_{-\infty} \Big(u v_t + F(u) v_x\Big)\,dxdt + \int^{\infty}_{-\infty} uv\,dx \big|_{t=\tau}= 0,\tag{3}
$$
for $v\in C^{\infty}_c(\mathbb{R}\times [\tau,\infty))$ which can be easily extended to the whole time domain smoothly. The difference between (2) and (3) is
$$
\int^{\tau}_{0} \int^{\infty}_{-\infty} \Big(u \color{blue}{v_t} + F(u) \color{blue}{v_x}\Big)\,dxdt + \int^{\infty}_{-\infty} gv\,dx \big|_{t=0} - \int^{\infty}_{-\infty} uv\,dx \big|_{t=\tau}= 0.\tag{4}
$$
Now you can either argue by (A) the test function makes (2) and (3) together must vanish on $[0,\tau)$, hence the blue terms are gone, or (B) choosing $v = 1$ on a set containing the support of $u$ from $t=0$ to $t=\tau$, and the support of $F(u)$ (notice $F\big(u(x\to \infty,t)\big) = F(0) = 0$, the compactly supportedness of $u$ implies that $F(u)$ is compactly supported), then blue terms are gone as well.
Hence we have by (4): for any $\tau>0$
$$\int^{\infty}_{-\infty} gv \,dx \big|_{t=0} =\int^{\infty}_{-\infty} uv\,dx \big|_{t=\tau}.$$
By the choice of the test function $v$ above in (B), we have
$$\int^{\infty}_{-\infty} g(x) \,dx =\int^{\infty}_{-\infty} u(x,\tau)\,dx.$$
