How to prove rigorously that you need $m \geq n$ equations with $n$ unknowns to be able to solve a system of equations? How to show that you need $m \geq n$ equations with $n$ unknowns to be able to solve a system of equations? I mean it seems obvious to me that you can't solve e.g. 2 equations with 3 unknowns, but how to prove that rigorously?
 A: Let's stick to linear systems over the reals, for the moment.
A linear system with $m$ equations and $n$ unknowns can be seen as an equation
$$
Ax=b
$$
where $A$ is an $m\times n$ matrix, $x$ is a (column) vector in $\mathbb{R}^{n}$ and $b$ is a (column) vector in $\mathbb{R}^m$.
We can also see $x\mapsto Ax$ as a linear map $f_A$ from $\mathbb{R}^n$ to $\mathbb{R}^m$; the statement “The system $Ax=b$ has a solution” is equivalent to the statement “$b$ is in the range of $f_A$”.
Let's assume $Ax=b$ has at least a solution and ask ourselves what we need for ensuring uniqueness of the solution.
So, suppose $Av=b$ and $Aw=b$, that is, $v$ and $w$ are solutions of the system. Then
$$
A(v-w)=Av-Aw=b-b=0
$$
which means $v-w\in\ker f_A$. Conversely, if $z\in\ker f_A$ and $Av=b$, then
$$
A(v+z)=Av+Az=Av+0=Av=b
$$
so $v+z$ is another solution to $Ax=b$.
Thus uniqueness of the solution is equivalent to $\ker f_A=\{0\}$.
The rank-nullity theorem ends the argument: we know that
$$
\dim\ker f_A=n-\operatorname{rank}A
$$
so $\ker f_A=\{0\}$ means $\operatorname{rank}A=n$. However, the rank of an $m\times n$ matrix also satisfies $\operatorname{rank}A\le m$. If we want that $\operatorname{rank}A=n$ we need also $m\ge n$.
Beware, though, that $m\ge n$ doesn't guarantee existence of a solution: the system
\begin{cases}
x+y=0\\
x+y=1\\
x+y=2
\end{cases}
has more equations than unknowns, but obviously it has no solution.
If you don't want to consider linear maps and rank-nullity, you can assume the system has been reduced into “row-echelon form” $Ux=c$ or, using the complete matrix, $[U \mid c]$. Existence of a solution is equivalent to the last column not being a pivot one. Assume this case: if $m<n$, there surely are free variables, because the number of pivots can't exceed the number of rows (that is, of equations), so the system will have infinitely many solutions.
Note that this is essentially a rephrasing of the rank-nullity theorem: the number of pivots in $U$ is exactly the rank of $A$ and the number of free variables is the dimension of the kernel of $f_A$.
In this analysis, being over the reals is the same as being over any field. Just the “infinitely many solutions” bit must be adjusted to “more than one” if the field is finite.

For general (algebraic) systems, there's no general answer. A system of two equations in three unknowns can still determine a unique solution: think to
$$
\begin{cases}
x^2+y^2=0\\
y^2+z^2=0
\end{cases}
$$
that has the single solution $x=0$, $y=0$, $z=0$ if we consider it over the reals. It will have infinitely many solutions if considered over the complex numbers.
A: We can consider the $n$ unknowns as forming a basis. Then, with $m\lt n$ equations, we can only create a proper subspace of the basis space.
A: You can solve 1 equation in two variables, for example $x^2+y^2=1$. The solution is a circle. Evenmore, you can have a unique solution, for example, in case of $x^2+y^2=0$! Just for systems of linear equations, underdetermined systems have either no solution or more than 1 solutions, as was explained in detail by egreg.
A: EDIT. Since no one seems to have read my post, I give some explanations. 
Let $K$ be an infinite field, $m>n$ and $A\in M_{n,m}(K):K^m\rightarrow K^n$. Then the affine equation $Ax=b$ has $0$ or an infinity of solutions, that is, cannot have a finite $>0$ number of solutions.
A one line proof: 
If $b\notin im(A)$ then $0$ solution, otherwise there is $x_0\in K^m$ s.t. $A^{-1}(b)=x_0+\ker(A)$.
What is surprising is that the previous result can be generalized to any polynomial function; obviously, the proof is much more difficult in the general case. We would like to prove the following:
Proposition. Let $K$ be an algebraically closed field, $f:\mathbb{K}^m\rightarrow \mathbb{K}^n$ be a polynomial function and $F$ be the fiber $f^{-1}(0)$. If $m>n$, then the set $F$ cannot have a finite $>0$ number of elements.
The previous result is true at least when $K=\mathbb{C}$. More generally, the following proposition and its proof are due to D. Barlet.
Proposition. Let $M,N$, be connected complex varieties of dimensions $m>n$, $f : M → N$ be a holomorphic function and $f(x_0)=y_0$. Then $x_0$ is not isolated in its fiber $f^{-1}(y_0)$.
Proof. We reduce to absurdity.
Part 1.  Let $p$ be the generic rank of $f$ . If $m = dim (M)>p$, then the fibers of the restriction of $f$ to $Z$, the dense open of $M$
on which $rank(f)=p$, are varieties of dimension $m − p > 0$; this comes from the constant rank theorem. Thus, the fibers associated to $Z$ (the generic points in $M$) are infinite.
Part 2. Lemma. If $f (x_0 ) = y_0$ and if $x_0$ is isolated in its fiber, then any point in a neighborhood of $x_0$ is isolated in its fiber. 
Proof. We may assume that $M= B(0, r) \subset \mathbb{C}^m$,
$x_0 = 0$, $N= B(0, R) \subset \mathbb{C}^n$, $y_0 = 0$. Since $0$ is isolated in $f^{-1}(0)$, there is $s$ s.t. $0 < s < r$ and $0\notin f (∂B(0, s))$. Let $U$ be the open connected component of $B(0, s) \setminus f^{−1} (f (∂B(0, s))$, containing $0$; let $V$ be the open connected component of $B(0, R) \setminus f (∂B(0, s))$, containing $0$; let $\tilde{f}:U\rightarrow V$ be the restriction of $f$ to $U$. $\tilde{f}$ is proper; indeed
if $K$ is a compact of $V$, then $f^{ −1} (K)$ is closed in $B(0, r)$ and does not intersect $∂B(0, s)$. Finally $f^{-1}(K)\cap U$ is a compact of $U$. Thus, for any $x\in U$, the fiber of $\tilde{f}$ in $f(x)$, is a compact analytic subset of $U\subset B(0,s)$. Consequently, this subset is finite and $x$ is an isolated point in the fiber associated to $f(x)$.  $\square$
Part 3. According to Lemma, the generic points in $M$ (the set $Z$), in a neighborhood of $x_0$, are also isolated in their fiber. According to Part 1., the generic rank $p$ of $\tilde{f}$ satisfies $m\leq p$; since $p\leq \inf\{m,n\}$ is always true, we obtain $m=p$ and finally $m\leq n$, a contradiction.
