Why do univariate indefinite integrals carry the $\mathrm {d}x$? I understand that the $\mathrm {d}x$ sort of comes from the $(x_i-x_{i-1})$ term in the Riemann sums of the integrals (just like the $\mathrm dg$ for Riemann-Stieltjes integrals), but when working with indefinite integrals, there are no Riemann sums, so there's no point in carrying a $\mathrm {d}x$ for any heuristic/pedagogic purpose.
Thus, why do we even add the $\mathrm dx$ in $\int f(x)\,\mathrm d x$? Just to explain the change of variables theorem (saying "$\mathrm dx=g'(u)\,\mathrm du$")?
 A: TL;DR: because $\newcommand{\d}{\mathrm d}\DeclareMathOperator{id}{id}\displaystyle\int$ is the inverse of $\d$, not $'$.

Let
$$\d f(x)=f'(x)\Delta x\tag{1}\label{eqn1}$$
for some linear increment $\Delta x$.
Also let $$\id:\:\mathbb R\to\mathbb R$$
be the identity function such that $$\id(x)=x;\tag{2}\label{eqn2}$$
plug $f=\id$ into $\eqref{eqn1}$ to see
$$\d\id(x)=\id'(x)\Delta x=\Delta x\tag{3}\label{eqn3}$$
(here the fact that $\id'(x)=1$ is being used); now take the LHS and RHS of $\eqref{eqn3}$:
$$\d\id(x)=\Delta x$$
and use $\eqref{eqn2}$ on the LHS to see
$$\d x=\Delta x,$$
so that $\eqref{eqn1}$ rewrites as
$$\d f(x)=f'(x)\,\d x.\tag{4}\label{eqn4}$$
Additionally, you can observe the following geometrical interpretation.
$\hskip{70 pt}$
Here $y=f(x)$, $\mathfrak T=f'(x)$ and $\d x$ is some arbitrary increment for which $\d y=\d f(x)$ provides the best linear approximation at point $P=x$.
Now, your question. The symbol $\displaystyle\int$ is defined as the inverse of the symbol $\d$:
$$\d\int\Phi=\Phi\tag{5}\label{eqn5}$$
for any $\Phi$. As with many, many other operations, the inverse here is multivalued (compare with $(\,\cdot\,)^2$, for which the inverse has two values: $-\sqrt{\,\cdot}$ and $\sqrt{\,\cdot\,}$), i.e. there generally are more than one $\int\Phi$ such that $\d\int\Phi=\Phi$, and incidentally you can prove that all of them differ by a constant, meaning that although $\d$ cancels with $\int$ cleanly when $\d$ comes first, we otherwise have that
$$\int\d\Phi=\Phi+C\tag{6}\label{eqn6}$$
holds for any constant $C$, as you can easily prove via differentiation of both sides, the fact that $\d C=0$ and $\eqref{eqn5}$ to cancel $\int$.
Taking the antiderivative may be represented with the use of the symbol $\displaystyle\int$. Let's say, we want the primitive $f$ of $f'$. Return back to $\eqref{eqn4}$:
$$\d f(x)=f'(x)\,\d x$$
and notice if we take the inverse of $\d$ of both sides:
$$\int\d f(x)=\int f'(x)\,\d x,$$
we can use $\eqref{eqn6}$ to cancel $\d$ out on the LHS, which gives us
$$f(x)+C=\int f'(x)\,\d x,$$
a very familiar identity.
Also, $\eqref{eqn4}$ may be of use when integrating to perform "$u$-substitution" without actually substituting anything:
$$I=\int x\cos\left(x^2\right)\,\d x=\frac12\int\cos\left(x^2\right)2x\,\d x=\frac12\int\cos\left(x^2\right)\,\d\left(x^2\right),$$
which just uses $\eqref{eqn4}$ to have $2x\d x=\d\left(x^2\right)$. You can now put $u=x^2$ if it pleases your eye:
$$I=\frac12\int\cos(u)\,\d u,$$
but this is in no way necessary, as you can (knowing that $\sin'=\cos$) directly go to
$$I=\frac12\int\cos\left(x^2\right)\,\d\left(x^2\right)=\frac12\sin\left(x^2\right)+C.$$
The chain rule also gets very intuitive with this notation.
A: Because $\int f(x,y)\, dx$ is different from $\int f(x,y)\, dy$
In more advanced text, I have seen $\int f$ being used.
A: Since the indefinite integral is defined as the set of antiderivatives of $f$, that is,
$$F: I⊆ℝ\rightarrow ℝ$$ such that $F$ is differentiable and it holds that:
$$\frac{dF(X)}{dx}=F'(X)=f(x) ∀ x\in I$$
 $$dF(X)=f(x)dx$$
then the symbol for the integral comes naturally. 
A: The primary reason for this is because integration is defined* on differential forms. In that context, we have that if $F'(x)=f(x)$ then $dF(x) = f(x)\,dx$. Intuitively, this is expressing that the change in $F$ is proportional to the change in $x$. That is, it marks that we take derivatives with respect to variables, not just in a vacuum. This has the convenient consequence that change of variables like $dx = g'(u)\,du$ are actually formal statements, with the usual notion of equality.
In a sense, $dx$ gives the right structure to $\mathbb R$, whereas if you integrated with a different differential form - like $u$ - you'd be integrating with respect to some kind of 'squished up' structure. For instance, with $du=2dx$ we get that our integrals $\int f(x)\,du$ come out twice as big as they're supposed to, because if we rewrite this, we see that $\int f(u/2)\,du$ - and we get $2F(u/2)+C=2F(x)+C$ because when we integrated with $du$, we made $x$ move slower than the variable $u$ that we integrated, so we counted everything too much. This isn't wrong, but it makes the point that our differential forms are telling us how much we 'weight' each point of the domain, and $dx$ is giving us the proper weighting.
This isn't to say that you couldn't omit the symbols and really just work on functions $\mathbb R\rightarrow\mathbb R$ in the obvious way - and in some contexts, it's useful to do this - but there is certainly some sense in including the $dx$ terms, as they do express something.
*There are sometimes other notions of integration. For instance, in measure theory, one often has that $dx$ specifies the measure you're using. That is, if you have something like:
$$\int f(x)\,dx$$
the function $f$ assigns values to each point in the space $\mathbb R$ and the term $dx$ says we integrate with respect to Lebesgue measure. This is as opposed to integrating with respect to, say, counting measure, where we just sum up the non-zero values of $f$, or any number of other measures. This is just a more general notion of weighting the domain.
