If there is $n$ equations, there must be $n$ unknowns... Why? Is this always the case and why is this the case? I would really like to find out more about it. To get a better sense of it.
Zhen said in one of the comments:

For the linear case it's a question of linear algebra. For the
  differentiable case, it's the implicit function theorem. I'm not sure
  what machinery we invoke for the general case...

Can someone elaborate on why is it a question of linear algebra for linear equations. And each of the others? Or at least, reference an outside source which could help me grasp the matter a little bit better?
 A: Ok, so a little introduction how to find parameters for the model. Suppose that you have a model
$$
f(x,k)
$$
where $k_1,k_2,k_3,...,k_n$ are unknown parameters. You would like to make some measurements $(x_i,f(x_i,k))_{i=1}^m$ to find all these parameters. Just an example:
$$
f(x,k) = k_1x+k_2.
$$
Here it is necessary and sufficient to make exactly two measurements at two different points $x_1,x_2$ to find both $k_1$ and $k_2$. With measurements I mean that given a single input $x_1$ you measure the output $f(x_1,k)$.
How does this procedure work? Suppose, you made inputs $x_1 = 0,x_2 = 1$ and obtain $f(0,k) = 0$ and $f(1,k)=1$. Then you have a system
$$
\begin{cases}
k_1\cdot0+k_2 = 0,
\\
k_1\cdot1+k_2 = 1
\end{cases}
$$
and as a result you have $k_1 = 1,k_2 = 0$. So, you solved the system of linear equations. This is the main idea: to find paramers, you make measurements and solve the system of equations.
Equations on paramteres only depend on how $f$ depends on $k$, and does not depend on how $f$ depends on $x$. If $f$ is linear in $k$, then you have to have exactly $n$ measurements unless you unluckily make some measurements which give you no new information, that will cause the degeneracy of system of linear equations.
In the general case, the number of measurements can vary. E.g. you may obtain an equation
$$
k_1^2+k_2^2 = 0
$$
which has a unique solution and does not ask for more measurements.
In other cases even $n$ measurements are not enough, this is what Implicit Function Theorem about. But usually (so, this is an informal statement) you need $n$ equations on $n$ variables to find all of them.
Finally, it's worth to mention that in real life your model is never perfect as well as the date form the measurements. So, given $2$ parameters you may obtain $100$ measurements which never can be satisfied by any pair of parameters. So to obtain the values of parameters, you give an estimation based, say on Least Squares method.
