Is there a big difference in having insufficient axioms and insufficient inference rules/proof procedure to have a complete theory?
It seems like in many cases adding a new inference rule or a new axiom has the same effect. For example, consider a language with 2-place connectives $\rightarrow, \land$. The language also has an inference rule $a\rightarrow b,a \Longrightarrow b$.
Now we can add a new axiom schemes: $a\land b \rightarrow a$ and $a \land b\rightarrow b$ for any formula $a,b$. An alternative is to add a new inference rules: $a\land b \Longrightarrow a$ and $a\land b \Longrightarrow b$. In this case I claim the theories are equivalent no matter the axiomatization.
In particular, I would think that in second order logic something prevents us from replacing inference rules with axioms, no matter how many inference rules are defined, since it is stated that second order proof calculus is always insufficient (for certain theories). Otherwise we could always use the same proof calculus and merely add axioms, and thus have a universal proof checking procedure. Why does adding axioms in place of inference rules fail?