how did they come up with polynomials as knot invariant I understand how to calculate a Jones polynomial for a given knot, but I am not sure why would one search for a polynomial for invariants. How did he come up with these calculation rules?
 A: Disclaimer: this is a historical account by someone who's not a historian, but who's looked at all these these primary sources over the years, and who might be misremembering some details.

Alexander found a polynomial invariant in the 1920s. The Alexander polynomial, in his original paper, was defined by computing a determinant of a certain matrix that is defined in a somewhat complicated way. In modern language, what Alexander did was take the Dehn presentation of the fundamental group of a knot to write a presentation matrix for the 1st homology group of the knot complement $X$ with local coefficients, where the local system is the one associated to $\mathbb{Z}[\pi_1(X)^{\mathrm{ab}}]$. The abelianization of a knot complement is always infinite cyclic, so we may regard $\mathbb{Z}[\pi_1(X)^{\mathrm{ab}}]$ as being $\mathbb{Z}[t,t^{-1}]$, the ring of Laurent polynomials in one variable. I'm not sure what Alexander knew about these ideas back then, since some key ideas in algebra and algebraic topology seem to still be at least ten years in the future, but it seemed like he had some intuition for what he was calculating (even if that intuition didn't explicitly make it into the paper), and in any case Alexander took the determinant of this matrix to get an element of $\mathbb{Z}[t,t^{-1}]$, then proved that this was an invariant of a knot (up to multiplication by $\pm t^{\pm 1}$).
In 1969, Conway found a "skein theoretic" definition for the Alexander polynomial, via his Conway potential. Alexander had actually mentioned a skein relation back in the original paper, showing that a knot with the three skein replacements had a linear relationship, since there's a way to get it using multilinearity of determinants. However, the skein-theoretic point of view was developed much farther by Conway.
My understand is that this development was inspiring to Kauffman, who then went back to Alexander's idea of using the Dehn presentation, but reinterpreted it in a formally statistical mechanical way, using matrix expansion tricks and his "clock theorem" to create a method to calculate Alexander polynomials by enumerating certain combinatorial structures associated to a knot diagram (see his Formal Knot Theory, 1983).
Just a year after that was published, Jones, who I understand wasn't thinking about knots at all, was studying statistical mechanical models (like the Potts model) using operator algebras. He showed some of the relations that certain operators satisfied to a colleague, who noted that they looked just like the braid relations for the braid group. He then proved that the trace of the operator gave an invariant not just of the braid, but of the knot given by its braid closure (it's a "Markov trace," meaning it's invariant under the moves for Markov's theorem from knot theory about knots being represented as braids). These operators are parameterized in one variable, and the trace happens to be a polynomial function in that variable -- hence, a polynomial invariant of knots.
Kauffman was able to quickly use the point of view he developed in Formal Knot Theory to come up with the Kauffman bracket, a graphical calculus for computing Jones polynomials. It, like his Alexander polynomial calculation, is also about enumerating certain combinatorial objects associated to a knot diagram, but interestingly the Kauffman bracket gives something that can be interpreted as a skein relation in an obvious way.
Very quickly after that, in 1985 Hoste, Ocneanu, Millet, Freyd, Yetter, Przytycki, and Traczyk semi-independently found the universal knot invariant satisfying a 3-term skein relation using the skeins appearing in the Alexander and Jones polynomials. It's not that they were looking for a polynomial invariant --- it just turns out that the entire space of such knot invariants based on such a skein relation end up being (Laurent) polynomial functions parameterized by two variables. In later work by quantum topologists, it's discovered that the HOMFLY-PT polynomial can be explained as interpolating knot invariants derived from $U_q(sl(n))$ quantum groups, with $q$ and $n$ corresponding (in some way) to the two variables of the polynomial.

If there's anything that can tie this story together and answer your question, maybe it's this: polynomial invariants have never seemed to have been the goal. Rather, investigations into other questions have led researchers toward invariants that happen to be polynomials, many times because polynomials are found to parameterize/interpolate entire families of other scalar-valued invariants.
A: When you have two knot embeddings (particularly complex ones), and you wish to know whether they are equivalent (fundamentally the same), you calculate the Jones or Alexander or... other knot polynomials for each.  If the polynomials are not the same, neither are the knots.
Alexander (then at the Institute for Advanced Studies in Princeton) was thinking about the knot equivalence problem and thought to apply algebra.
