Im at that stage in my mathematical understanding where I kinda understand what set theory is and what first order logic is but dont really understand how they fit together to create Mathematics. I assume that the ZF system uses first order logic to create the foundations of mathematics and in the grand scheme of things, set theory is dependent on logic for its existence whereas logic or any formal system can exist on its own. Is this the correct view?
The received wisdom is that pretty well any mathematical statement can in principle be formulated and hence formalised in ZF. I think this is rather an overstatement, but let's bear with it. So in some sense, logic can be viewed as providing a formal foundation for mathematics. However, to do logic rigorously, you need to be able to define and reason about syntax, so a certain amount of mathematics is required to underpin logic. Many people take the view that the finitistic mathematics you need to reason about syntax is sufficiently certain that the apparent circularity is harmless. (Personally, I am agnostic on nearly all such issues.)
Not the best answer, but I'll try with a short answer. Some of the set theory books, you will find it says for example $A\subseteq A\cup B$, where $A$ and $B$ are both sets. If we want to check if it's true, we have to prove that the statement $(x\in A)\implies (x\in A\cup B)$ is true. Note that this is equivalent to $(x\in A)\implies [(x\in A)\vee (x\in B)]$, and it's actually true. The reason why it's true is because the statement $p\implies (p\vee q)$ is a tautology which can be read more in Mathematical logic. There is a nice table that shows different tautologies in Rules of Inference. The rule we used here is called "Addition".
From my understanding, you can use set theory to model the objects studied in mathematics. Then major mathematical theorems can be proven from these models. Take for example, ZFC. If you accept the axioms, then you can use the language of first-order logic to deduce theorems. In Enderton's book on set theory (Elements of Set Theory), he gives a very easy to understand construction of the real numbers from these basic axioms. He first constructs the natural numbers, then defines ordered pairs and equivalence classes using sets. This allows him to construct integers, then rationals, and finally real numbers. Now we can prove theorems about real numbers and eventually theorems in calculus. So in this sense, first-order logic describes the rules of the language in which the axioms of set theory are written, and how rules of inference can be applied to these axioms to create theorems. And since we can use sets to model a lot of things in mathematics, we can say that they can be used as a foundation for mathematics.