The question is strange and unfocussed but suggestive, so I'll respond to some points.
I think that the starting point for Euclid was in fact something like your "define the basic notions with a physical reference". The Greeks thought they knew what points and lines were, so managed to build geometry with essentially undefined primitives. They also thought they knew what the Euclidean plane was, hence the subsequent controversy over the parallel postulate, which mathematicians felt for centuries ought to be a theorem.
In time Gauss, Bolyai and Lobachevsky (independently) discovered that there were consistent geometries in which the parallel postulate failed. That led over time to the view that mathematics wasn't really about anything. As long as it was contradiction free it was OK. In that sense mathematics is just "one big circular reference". That said, most mathematicians believe in their day to day work that the objects they are proving theorems about are quite real, and don't feel the need to justify that belief (so back to the Greek model, in a way). They don't even worry often about the fact that Godel proved that you couldn't prove everyday mathematics was free of contradictions.
As for the dictionary, think of it as a big directed graph, with nodes the words and edges the assertion that one definition depends on another. Then in some sense the meaning of the language is the whole graph, which is more than the collection of words. You can't think of just the words as the primitives.
Computer programs are used to facilitate translations and to decode messages and ancient languages. The best of those programs deal with the contexts in which they find words, not just with one to one correspondences between words, so graph theory is again in play.