A structural view to the power set axiom: Is this axiom really justifiable? The power set axiom in set theory states that the collection of the subsets of a set is a set itself. 
I wonder if this is a "natural" axiom in the sense that if we consider sets as the simplest structures in mathematics and replace the word "set" in power set axiom with "structure" of arbitrary type, we get the following:

The collection of sub-structures of a given $\mathcal{L}$ - structure $\mathcal{M}$ form an $\mathcal{L}$ - structure. ($\mathcal{L}$ is a first order language. However it is not essential to restrict ourselves to the first order framework).  

Then we can ask the following question:

Question. In what realms of mathematics do the substructures of a fixed structure form a structure of the same type? For example is there any "natural" (whatever it means) group structure on the collection of all subgroups of a given group $G$? Or does sub-spaces of a given vector space $V$ form a vector space? What about finding a field structure on the sub-fields of a field?

Existence of such phenomenon in other parts of mathematics might be considered as a kind of justification for validity of the power set axiom in its structural form.
 A: Ignoring the philosophical trappings (e.g. justifying the powerset axiom), there are two questions here, as I see it: "Can we naturally define an $L$-structure on the powerset of an $L$-structure?" and "Does that resulting 'hyperstructure' resemble the original?" I believe the answers are "Yes" and "Usually not," respectively:
If $L$ is a language consisting entirely of function symbols - e.g., the language of groups, with a nullary function (=constant) for the identity, a unary function for the inverse, and a binary function for the group operation - then given any $L$-structure $\mathcal{A}$, we can view $\mathcal{P}(\mathcal{A})$ as an $L$-structure in a natural manner: for each unary function symbol $f\in L$, we define $$f(X)=\{f(x): x\in X\},$$ and more generally for an $n$-ary function symbol $f\in L$ we define $$f(\overline{X})=\{f(\overline{x}): x_i\in X_i\}.$$ So we can now ask:

When is it the case that, if $\mathcal{A}$ has property $p$, then so does $\mathcal{P}(\mathcal{A})$?

For example, if $\mathcal{A}$ is a monoid, it's easy to see that $\mathcal{P}(\mathcal{A})$ is also a monoid. However, the same is not true for groups, most obviously because the emptyset annihilates all functions. Note that this means that even equational properties are not preserved by powerset. So this question operates at a pretty fine-grained level.
I have no idea what a full answer to that question is, but note that the annihilating properties of the emptyset kill off a lot of possibilities: the powerset of a group is not a group, the powerset of a field is not a field, etc.
A: While I think that Noah's answer is closer to what you are looking for, I'd like to discuss your question in a very different way. Given any first order $L$-structure $\mathcal A$ with infinite universe $A$, the Löwenheim-Skolem Theorem enables us to construct an $L$-structure $\mathcal A^*$ with universe $\mathcal P(A)$ that satisfies the same $L$-theory. So, if $\mathcal A$ is an abelian group, then $\mathcal A^*$ is an abelian group as well. If $\mathcal A$ is a dense linear order, then so is $\mathcal A^*$, etc.
We can even construct $\mathcal A^*$ such that for all $a_1, \ldots, a_n \in A$ and any $L$-formulua $\phi$ with free variables $v_1, \ldots, v_n$ the following holds:
$$
\mathcal A \models \phi(a_1, \ldots, a_n) \text{ iff } \mathcal A^* \models \phi( \{a_1\}, \ldots, \{a_n\}),
$$
i.e. $\pi \colon A \to \mathcal P(A), a \mapsto \{a\}$ is an elementary embedding from $\mathcal A$ to $\mathcal A^*$. Consequently, function and relation symbols are respected as well, when restricted to the image of $\phi$, i.e. for all relation-symbols $R$ in $L$ and all $a_1, \ldots, a_n \in A$ we have
$$
(a_1, \ldots, a_n) \in R^{\mathcal A} \text{ iff } (\{a_1\}, \ldots, \{a_n\}) \in R^{\mathcal A^*}
$$
and for all function symbols $f$ in $L$ and all $a_1, \ldots, a_n \in A$ we have
$$
f^{\mathcal A^*}(\{a_1\}, \ldots, \{a_n\}) = \{ f^{\mathcal A}(a_1, \ldots, a_n) \}.
$$
In the special case that $f=c$ is a $0$-ary function symbol, i.e. a constant symbol, we get that
$$
c^{\mathcal A^*} = \{ c^{\mathcal A} \}.
$$
This may be viewed as a (somewhat) natural extension of $\mathcal A$'s structure to $\mathcal P(A)$ and restricted to the singletons $\{a \} \subseteq A$ even agrees with Noah's construction.
