Generally the compact set of premises isn't the starting point. It's something that is produced as part of the organisation of our knowledge. When we try to understand how the things we know relate to one another, how certain things logically follow from other things, what are the things we base our arguments upon, we find that there are some things that we cannot really explain to ourselves in terms of anything more simple or more obvious. We can only identify and collect these things, and show how everything else we know is derived from them.
But the resultant organisation of our knowledge as being wholly derived from these few fundamental intuitions shouldn't be mistaken for the historical process of how our knowledge actually came to be. A mathematical proof is not a record of either how the idea of a theorem formed, or the process by which the proof was found, and the body of organised mathematical theory does not represent the process by which mathematics is created.
The real basis of mathematics is our natural informal intuitions of number, distance, area, volume, direction, rhythm, more than/less than/the same, inclusion/exclusion, adding/taking away, multiplicity, division, shared/unshared properties, argument, straightness, inside/outside, etc, etc, etc.
Mathematics: The Loss of Certainty is a good book to read about the history of trying to set the foundations of mathematics. Suitable for undergraduates too.