Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is this always the case and why is this the case? I would really like to find out more about it. To get a better sense of it.

Zhen said in one of the comments:

For the linear case it's a question of linear algebra. For the differentiable case, it's the implicit function theorem. I'm not sure what machinery we invoke for the general case...

Can someone elaborate on why is it a question of linear algebra for linear equations. And each of the others? Or at least, reference an outside source which could help me grasp the matter a little bit better?

share|cite|improve this question
I presume you've looked up overdetermined and underdetermined systems? – J. M. Sep 23 '11 at 11:16
Maybe the geometric approach might be of use to you: you're aware of "two (non-parallel/non-coincident) lines cross at only one point"? The bit about needing $n$ equations in $n$ unknowns is but a natural generalization... – J. M. Sep 23 '11 at 11:22
No, it is not always the case that if there are $n$ equations there must be $n$ unknowns. For example, here is a set of 2 equations in 1 unknown: $x+1=2$ and $x+2=4$. And here is a set of 2 equations in 3 unknowns: $x+y+z=10$, $x-y+z=20$. (It is a different matter that if you have more equations than unknowns then the system may be overdetermined and have no solutions, and if there are more unknowns than equations the system may be overdetermined and have many solutions, but a set of equations can have any number of unknowns in it.) – ShreevatsaR Sep 23 '11 at 11:37
up vote 3 down vote accepted

Ok, so a little introduction how to find parameters for the model. Suppose that you have a model $$ f(x,k) $$ where $k_1,k_2,k_3,...,k_n$ are unknown parameters. You would like to make some measurements $(x_i,f(x_i,k))_{i=1}^m$ to find all these parameters. Just an example: $$ f(x,k) = k_1x+k_2. $$ Here it is necessary and sufficient to make exactly two measurements at two different points $x_1,x_2$ to find both $k_1$ and $k_2$. With measurements I mean that given a single input $x_1$ you measure the output $f(x_1,k)$.

How does this procedure work? Suppose, you made inputs $x_1 = 0,x_2 = 1$ and obtain $f(0,k) = 0$ and $f(1,k)=1$. Then you have a system $$ \begin{cases} k_1\cdot0+k_2 = 0, \\ k_1\cdot1+k_2 = 1 \end{cases} $$ and as a result you have $k_1 = 1,k_2 = 0$. So, you solved the system of linear equations. This is the main idea: to find paramers, you make measurements and solve the system of equations.

Equations on paramteres only depend on how $f$ depends on $k$, and does not depend on how $f$ depends on $x$. If $f$ is linear in $k$, then you have to have exactly $n$ measurements unless you unluckily make some measurements which give you no new information, that will cause the degeneracy of system of linear equations.

In the general case, the number of measurements can vary. E.g. you may obtain an equation $$ k_1^2+k_2^2 = 0 $$ which has a unique solution and does not ask for more measurements.

In other cases even $n$ measurements are not enough, this is what Implicit Function Theorem about. But usually (so, this is an informal statement) you need $n$ equations on $n$ variables to find all of them.

Finally, it's worth to mention that in real life your model is never perfect as well as the date form the measurements. So, given $2$ parameters you may obtain $100$ measurements which never can be satisfied by any pair of parameters. So to obtain the values of parameters, you give an estimation based, say on Least Squares method.

share|cite|improve this answer
Just a comment, motivated by @Shree's comment above: That you can have fewer equations follows since you can insert a "false" equation into the (deficient) mix and end up with no solutions. That you can have more equations follows since you can insert any number of "true" equations (lin. dep.) after the n-th equation. – The Chaz 2.0 Sep 23 '11 at 12:19
@TheChaz: Yes, I've just tried to give an observation how does this problem arise. – Ilya Sep 23 '11 at 12:26

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.