Note that before talking about theories and provability we need to first select a language to express formulas/statements. Here the language is the language of first-order logic.
A language is just a set of symbols and how we can combine those symbols to create meaningful expressions (well-founded formulas).
After selecting a language we can start talking about proof systems (mathematical reasoning systems). A proof system is an (efficient) algorithmic way of checking if a given string is a proof of a given formula (encodes as a string). Think of it as an algorithm that getes two inputs $P(\pi,\varphi)$ (a binary computable predicate) which is true if and only if string $\pi$ is a proof of formula $\varphi$ in $P$. People sometimes write $\pi : P \vdash \varphi$ or omit $\pi$ and write $P \vdash \varphi$. Often the $P$ is put below the symbol like $\vdash_P \varphi$, and when it is clear from the context it is omitted completely $\vdash \varphi$.
What is $P$? We would like that $P$ captures our intuitive notion, i.e. we would like it to satisfy the following conditions:
soundness: $P$ accepts $\pi$ and $\varphi$, then $\varphi$ is true.
completeness: if $\varphi$ is true, then $P$ accepts $\pi$ and $\varphi$.
The conditions above presume a notion of semantic truth and are what ties a syntactic objects like proofs to the semantic notion of truth. For validity in first-order logic we can satisfy these conditions, i.e. first-order logic has a sound and complete proof system. However this is not the case for stronger systems like PA or ZF where by incompleteness theorem there is no way of algorithmicly enumerating the true statements (in dependent of the way you define the truth as long as we agree that some basic statements are true, e.g. Robinson's arithmetic theory $Q$).
It is usual in mathematics to take $P$ to be proofs in first-order logic + some first-order theory. ZF is one of such theories. William explains some reasons for this in his answer. A key point here is that first-order logic is very simple and intuitive. It is simpler than the amount of mathematics that one needs to do the stuff I explained above. It is seldom disputed (at least not classical mathematics). By separating the first-order logic part from the first-order theory part we can focus on the disputed part of reasoning. There are mathematicians who would not accept ZF and but you will have a difficulty finding anyone who doesn't accept first-order logic.
If you remove the axioms for first-order logic, then you cannot prove even simple things because the axioms for first-order logic are the part that give us the meaning of the symbols. For example, we have axioms $P \land Q$ implies $P$, $P \land Q$ implies $Q$, and $P$ and $Q$ imply $P \land Q$. If you don't have these axiom, then meaning of the symbol $\land$ can be arbitrary. There is nothing forcing it to act as "and" other than these axioms. Same applies to other axioms of first-order logic. I would suggest that you have a look at a proof system for first-order logic to understand (e.g. LK) to understand that the meaning and use of the symbols is a result of these axioms and without them the symbol is meaningless.
Let me add something that confuses people who are new to mathematical logic. Logic is not build on nothing! We are already assuming a considerable amount of arithmetic in manipulating symbols. For example, if $f$ is a function symbol and $x$ is a variable, we assume that we can create $f(x)$, $f(f(x))$, etc. and this can be views as similar to the successor function over natural numbers. The minimum amount that is needed not too much, but it at least will contain a form of induction over natural numbers. The usual theory that people assume for this meta theory when talking about strong theories is PRA (but there are weaker theories that are sufficient, the bounded arithmetic theory VTC is the weakest meta-theory that I know which allows relatively easy development, see "Logical Foundations of Proof Complexity" for more detail.) Also see Petr Hajek and Pavel Pudlak's book "Metamathematics of First-Order Arithmetic".