Are $\Bbb{P} (A \rightarrow B)$ and $A \rightarrow \Bbb{P}(B)$ related?! Let $A \rightarrow B$ be the set of all functions from $A$ to $B$, and let $\Bbb{P}(A)$ be the set of all probability distributions on A, i.e.
$p : A \rightarrow [0, 1]$
subject to the usual restrictions (e.g. $a \in A ,\ \Sigma p(a) = 1$ in the discrete case). What is the relationship between
$\Bbb{P} (A \rightarrow B)$   and   $A \rightarrow \Bbb{P}(B)$?
I believe $\Bbb{P} (A \rightarrow B)$ and $A \rightarrow \Bbb{P}(B)$ are related in some ways but I cannot find any relation between them. 
Any comments are appreciated, thanks.
 A: As a disclaimer, I plan to sweep a lot of technical details under the rug - really, we should be defining $P$ to take measurable spaces $(A,\sigma)$ to the set of probability measures thereon equipped with another sigma algebra. This is a real pain and largely obscures the more conceptual ideas, so I will largely ignore it - but the reader should know that these details do actually work out.

To start with, letting $P(X)$ be the set of all probability measures on $X$, it's worth observing that there is an interpretation for one of the sets you are interested in:

A stochastic function from $A$ to $B$ is a function $A\rightarrow P(B)$. We may interpret such a map to mean that, for each input $a\in A$, we choose some $b\in B$ possibly according to some randomized rule.

For instance, a simple example to consider would be to consider a stochastic function $f$ from $\mathbb Z$ to $\mathbb Z$ which takes $n$ to either $n-1$ or $n+1$ with equal probability - or, formally, that takes $n$ to the probability measure $\frac{1}2\left(\delta_{n-1} + \delta_{n+1}\right)$. This defines a single step of a random walk.
There's an interesting problem with this definition: intuitively, you should be able to compose stochastic functions - for instance, what if we wanted to model two steps of a random walk? Well, we'd want to write $f\circ f$ using the previous function $f$ - but that doesn't work because $f$ is a map from $\mathbb Z$ to $P(\mathbb Z)$, so the domain of the first function applied doesn't match up to the domain of the second function applied!
Generally, if we have stochastic maps $f:A\rightarrow P(B)$ and $g:B\rightarrow P(C)$, we'd like to be able to compose them, but we can draw out the problem visually:

We have two maps $f$ and $g$ labelled, but what we really want is the dotted map - but simple function composition won't get there! This does start to hint that we should want maps $B\rightarrow P(C)$ to somehow be related to maps $P(B)\rightarrow P(C)$, because if we could fill in an arrow there, we would be able to carry out the composition as we desire.
For this discrete case, the rule that we want to come up with is loosely stated as follows:

The probability that $(f\circ g)(x) = y$ is the sum, over all $z$, of the product of the probabilities that $g(x) = z$ and $f(z) = y$.

One can just extend this rule to define a composition directly - but there's some deeper structure here:
First, $P$ is a functor - that means that it not only takes spaces $X$ to spaces $P(X)$, but it also can transform a map $A\rightarrow B$ into a map $P(A)\rightarrow P(B)$. In particular, if you have a function from $f:A\rightarrow B$, we define $Pf:P(A)\rightarrow P(B)$ to essentially be the function that, given a measure $\alpha$ on $A$, chooses some random $a$ according to $\alpha$ and maps it to $f(a)$. Otherwise said, if $S\subseteq B$, then $Pf(\alpha)$ is a probability measure where the measure of $S$ is $\alpha(f^{-1}(S))$.
This gets us closer, because now we can see that we can definitely compose $f$ with $Pg$, and that seems to get us closer:

The remaining issue is that this is now a map from $A$ to $P(P(C))$, which is no good!
However, we can then make an observation: there is a map from $\mu_C:P(P(C))$ to $P(C)$ which is basically given by the phrase "Given a distribution $\alpha$ on $P(C)$, choose a random distribution $\beta$ according to $\alpha$ and then a random $c$ according to $\beta$." Formally, one ends up with the following definition:
$$\eta(\alpha)(S) = \int \beta(S)\,d\alpha(\beta).$$
This solves our problem! We can stick $\mu_C$ into our diagram, and we get

where now we can compose $\mu_c \circ Pg \circ f$ to get a stochastic function from $A$ to $C$ - as we desired at the start. One may check that this truly does give the function we had desired.
Okay, so now we can return to the original question: What is the relationship of $B\rightarrow P(C)$ and $P(B)\rightarrow P(C)$ then?
Well, we just showed that, given any function $g:B\rightarrow P(C)$, we can turn it into a function $g':P(B)\rightarrow P(C)$ given as $g' = \mu_C \circ Pg$ which represents how we can apply a stochastic function to a distribution, just by randomly picking a point according to that distribution and then getting the output via the function. This is important to being able to compose functions.
There's also another way these two sets are related: for any $B$, we can define $\delta_B$ to be a function $B\rightarrow P(B)$ which takes $b\in B$ to the unit mass $\delta_b\in P(B)$. Using this, we can also get turn a function $h:P(B)\rightarrow P(C)$ into a function $h':B\rightarrow P(C)$ by the rule $g=g'\circ \delta_{B}$, which corresponds to evaluating $g'$ at the unit mass associated to each point.
Thus, we end up with ways to transform function $A\rightarrow P(B)$ into functions $P(A) \rightarrow P(B)$ and vice versa - which is a very intimate and useful link between the two sets. One can summarize the data we've found in a single diagram:

Then, to conclude, it's worth noting how these two transformations play together. In particular, if you start with a function $g:A\rightarrow P(B)$ then define $g':P(A)\rightarrow P(B)$ as $\mu_C \circ Pg$ and then return to a function $A\rightarrow P(B)$ by defining $g''$ as $g'\circ \delta_B$, we will find out that $g'' = g$. Equivalently said, the above diagram commutes for every $g$, which is a very special property. Starting with a function $P(A)\rightarrow P(B)$ then transforming it twice does not give you the same function back - which tells you, more or less, that maps $P(A)\rightarrow P(B)$ are actually more general than those $A\rightarrow P(B)$.
The really cool thing about this relationship is that it allows us to compose these stochastic functions $A\rightarrow P(B)$ in a sensible way; one may check that composition defined this way is actually associative and that $\delta_A:A\rightarrow P(A)$ acts like the identity function in composition - which tells you that the definitions of $\delta_A$ and $\mu_C$, aside from capturing the right intuition, satisfy some nice relationships. In much greater generality, $P$ is an example of a monad, which is a category theory term that captures the essentially the structure discussed here - and which is almost tailored to solve the problem of "I have these maps $A\rightarrow P(B)$ and $B\rightarrow P(C)$ and I want to compose them."
