Preamble: This is a very general question, which I have always been wondering about. I understand there might not be well formulated way of developing algorithms I just would like to seek and see different opinions on this.
The reason I am posing this on MATHEMATICS stack exchange is because I feel some how it relates to the difference between applied and pure mathematics.
Question: It seems developing different algorithms (techniques) to find a solution of a problem is normally done via trial and error, and sometimes based on some intuitions, but it never involves as much rigor as pure mathematics does. Take the techniques to find the solutions of polynomial equations for example. These were found and used by early mathematicians way before the establishment of modern rigorous mathematics (history of solution of polynomial equations).
Or another example could be the ML estimator which was used way before axioms of probability was proposed by Kolmogorov. It is a very intuitive thing that we take the maximum of probability distribution of the observation as an estimate of the desired parameter. ML might have been studied more rigorously in modern probability theory by now though.
Or even some methods and techniques used in calculus like integration, where they have their roots in ancient Greek mathematics. Again way before calculus was formally studdied.
So the question is that: is this same approach used today by modern mathematicians to develop techniques based on intuition and trial and error Or is there a more formulated way to find solutions and develop algorithms?
Finally, does the pure mathematics tells us something or show us something (for example a polynomial equation has a solution) and then is the purpose of applied mathematics (or engineering) to find solution and algorithms using experimentation?