Every field of knowledge involves concepts. Sciences, and especially mathematics, require high precision through carefully formulated concepts. The dictionary phenomenon is that concepts are explained by simpler concepts and hence the simplest (so-called primitive) concepts have to be left unexplained. In set theory, we have primitive concepts like "set" and "membership". In geometry we have primitive concepts like "point", "line" and "incidence", etc.
The axiomatic method of mathematics is a wonderful and pretty unique way of getting around this problem: the basic idea is not attempting at defining primitive concepts but prescribing how they (inter)relate. In practice, this is exactly what you need to deal with them. For instance, projective geometry requires (among other things) that every two distinct lines have a unique common coincident point and every two distinct points have a unique coincident line. Set theory requires (among many other things) a set with no members (empty set) and with every set A and every statement P(x), a set with members: all members x of A satsifying P(x).
In this way, a beautiful method obtains to express primitive concepts or mutually dependent concepts (like set and membership) accurately and elegantly. Axioms are also the starting point of logical deduction. In a way, mathematics is information processing with logic, the information being about concepts.