Uniqueness of primitives We all know that a primitive $F(x)$ of a function $f(x)$ is the function which derivative is $f(x)$. For example:
$$\int 2x dx = x^2 + C$$
where $x^2 + C$ is the set of all primitives of the function. However, how can I be sure that $x^2 + C$ is the only possible solution? How can I be sure that a function $f(x) \ne x^2 + C\ ,\ f'(x) = 2x$ does not exist?
How could I create a mathematical proof of this fact (in this specific case and in the general case)? Does it make sense to seek such a proof? If not, why?
 A: What you said is a consequence of the following:
If $f^{\prime}=g^{\prime}$ then $f=g+c$ for some constant $c\in \mathbb{R}$. 
To prove this let $h=f-g$. Then $h$ is differentiable and $h^{\prime}=f^{\prime}-g^{\prime}=0$. $h$ is therefore constant (*), that is $\exists c\in \mathbb{R}:h(x)=c$. Then, $f(x)=g(x)+c$
Therefore, a continuous function always has a primitive, unique up to additive constant.
(*) This is an important consequence of the Mean Value Theorem.
Here is a proof of that
Let $f$ be continuous in $[a,b]$ and differentiable in $(a,b)$ so that $f^{\prime}\equiv 0$. Let $x,y\in [a,b]$ and WLOG $x<y$. By the MVT in $[x,y]$,
\begin{equation}\exists \xi \in (x,y):f^{\prime}(\xi)=\frac{f(y)-f(x)}{y-x}\implies\frac{f(y)-f(x)}{y-x}=0\implies f(y)=f(x)\end{equation}
and so $f$ is constant
A: Your question is eminently sensible and indeed important.
The keyword here is Mean Value Theorem, or more specifically its special case that a function defined and differentiable on an interval with identically zero derivative must be constant.  For more details, see (e.g.) $\S 5.1$ of these notes.
Here are some further remarks:
$\bullet$ Of course it is understood here that all functions are defined on an interval $I$.  If we look at functions with more complicated -- and in particular disconnected -- domains, then uniqueness of primitives up to a constant need not hold.  One sees the ugly head of this in freshman calculus when students are taught (not by me!) that the antiderivatives of $\frac{1}{x}$ are $\log |x| +C$.
$\bullet$ The fact that a function which is differentiable on an interval with identically zero derivative is constant -- I call this the Zero Velocity Theorem -- is actually quite deep.  As mentioned above, it is a consequence of the Mean Value Theorem.  In turn it holds in an ordered field iff the field is Dedekind complete, as is shown in Jim Propp's nice article Real Analysis in Reverse.  It is easy to construct counterexamples over $\mathbb{Q}$: just take a piecewise constant function with discontinuities at irrational numbers.
$\bullet$ A proof of the Zero Velocity Theorem directly from an equivalent of the least upper bound axiom -- specifically, using Real Induction -- is given here.
A: Let $F(x)$ be a function such that $F'(x)=2x$ for all $x$. Let $G(x)=F(x)-x^2$. Then $G'(x)=0$ for all $x$.
Let $G(0)=C$. By the Mean Value Theorem, for any $x\ne 0$ there is a $t_x$ between $0$ and $x$ such that
$$\frac{G(x)-G(0)}{x-0}=G'(t_x).$$
Since $G'(x)$ is identically $0$, it follows that $G(x)-G(0)=0$, that is, that $G(x)=C$. Thus $F(x)=2x+C$ for all $x$. 
Remark: Let us find all the antiderivatives of $\dfrac{1}{x^2}$. Here the correct answer is not quite the conventional one. By the argument above, one can show that if $F'(x)=\dfrac{1}{x^2}$ then there is a constant $C$ such that  for all positive $x$, we have $F(x)=-\dfrac{1}{x}+C$. 
Similarly, there is a constant $D$ such that   for all negative $x$, we have $F(x)=-\dfrac{1}{x}+D$.
But the constants $C$ and $D$ need not be equal. The singularity at $x=0$ prevents the derivative from transferring information from the positives to te negatives.  
