The power in the power set axiom is the ability to create larger sets than any other axiom is capable of. At least we want it because we probably want $\mathbb R$ (to be a set).
The other axioms doesn't seem to be strong enough to guarantee the existence such large set (larger than $\mathbb N$).
Note that dropping an axiom would not make it (automatically) false, because if it could be proven to be false when dropped we could use the same proof in a set theory where we don't drop it - and thereby get a contradiction.
In order to get a set theory where we're guaranteed to have a set without a power set we would actually have to replace the axiom of power set with it's negation (saying that there's a set without a power set). The possibility to do so without contradiction requires that the axiom is independent of the others (that is it can't be proven from the other).
To see what the axioms are able to produce out of the set we supply (that is what we already have) let's pick them one by one:
Axiom of Extensionality: Doesn't guarantee any existence at all (it's only stating a relation between supplied sets).
Axiom of Regularity: It hardly produce anything new, it only guarantee properties of the sets inside any set.
Axiom schema of Specification: It will only produce smaller sets, the produced set is a subset (with elements having the property) of the supplied set.
Axiom of Pairing: It will produce sets of size one or two given a supplied set.
Axiom of Union: It will produce the union of supplied sets, this will at most have cardinality of the outer set multiplied with the highest cardinality of the contained set. At most we can square cardinalities by this.
Axiom schema of Replacement: It will produce a set of lower cardinality, however the cardinality of it's element can be any cardinality we can construct. This means we can't increase cardinality in any way here.
Axiom of infinity: This is allowed to produce an infinite set out of nothing. Not only can we create an infinite cardinality, but we also can create sets which can be supplied to the other axioms. No particular infinity are not specified in the axiom, just that it has at least cardinality $\aleph_0$.
Well ordering axiom: This can construct a total order on a set. The cardinality of a total order on $A$ is $|A|(|A|+1)/2$. At infinite cardinalities this means squaring the cardinality.
Axiom of Power set: This will create power sets of any set $A$ which has the cardinality $2^{|A|}$.