How would you state categorically that $\mathcal{P}(X)$ and $2^X$ contains the same information? How would you state, in category theory, that the set of all subsets of a given set $X$, contains the same information as the set of all functions with domain $X$ and codomain $2$?
Note that we define $2$ as $\{0, 1\}$, see von Neumann ordinals.
I'm assuming that one would define some sort of natural transformation.
 A: One possibility is stating that $2$ is the subobject classifier of the category of sets. You can have a look at the wiki or the nlab. Indeed you can see the statement as establishing a certain (natural) isomorphism of the functors $\mathrm{Sub}_{\mathrm{Set}}$ and $\mathrm{Hom}(-,2)$.
(I was going to post this as a comment but I guess it's more like an answer?).
A: Your intuition is correct. I would phrase this as saying there is a natural isomorphism from the functor
$$\operatorname{Hom}({-}, 2) : \mathbf{Set}^\mathrm{op} \to \mathbf{Set}$$
to the functor
$$\mathcal{P} : \mathbf{Set}^\mathrm{op} \to \mathbf{Set}.$$
Note that there is more than one way to make the assignment $X \mapsto \mathcal{P}(X)$ into a functor. Here, I'm using the way that sends a function $f : X\to Y$ to the inverse image operation $\mathcal{P}(f) = f^{-1} : \mathcal{P}(Y) \to \mathcal{P}(X)$.
Exercise: What's another way to make the assignment $X\mapsto \mathcal{P}(X)$ into a functor?
Exercise (if you are familiar with Yoneda lemma): The Yoneda lemma tells us that natural transformations $\operatorname{Hom}({-}, 2) \to \mathcal{P}$ correspond to naturally to elements of $\mathcal{P}(2)$. Identify which subset of $2$ witnesses the natural isomorphism you want.
