I saw some result that some article used, (without proving) that stated:$$\int_0^1 p^k (1-p)^{n-k} \mathrm{d}p = \frac{k!(n-k)!}{(n+1)!}$$
But I was wondering, how would you integrate it? How did this integral come about? Is it something to do with the binomial distribution?
Not a natural derivation, but there is slightly different approach toward it.
Let's consider the quantity
$$I(n,k) = \int_{0}^{1} \binom{n}{k} p^k (1-p)^{n-k} \; dp.$$
Then by integration by parts, as in two former answers, we have
$$I(n, k+1) = I(n, k).$$
Let $I$ denote this common value. Thus
$$1 = \int_{0}^{1} 1 \; dp = \int_{0}^{1} \sum_{k=0}^{n} \binom{n}{k} p^k (1-p)^{n-k} \; dp = \sum_{k=0}^{n} I = (n+1)I$$
and the result follows.
You can also proof it "by story". Let "random number" mean a number picked from $[0,1]$ with uniform probability. Then the formula below can be interpreted as follows.
$$\int_0^1 p^k (1-p)^{n-k} \mathrm{d}p = \frac{k!(n-k)!}{(n+1)!}$$
The left-hand side is the probability of taking random $p$ and then drawing a sequence of $n$ numbers from which some $k$ numbers are smaller than $p$ and some $n-k$ are larger.
To understand the right-hand side, consider $n+1$ random numbers sorted, so that first $k$ are the smallest and last $n-k$ are the largest (with $(k+1)$-th being our $p$ from left-hand side interpretation); however, there are $(n+1)!$ permutations total, with $k!(n-k)!$ having the desired property (in a sorted sequence we disregard the order of first $k$ and last $n-k$), thus the right-hand side fraction denotes the same probability.
There are probably several ways. An easy one is by induction on $k$.
If $k=0$, then $$ \int_0^1(1-p)^n\,dp=\left.-\frac{(1-p)^{n+1}}{n+1}\right|_0^1=\frac1{n+1}=\frac{0!(n-0)!}{(n+1)!}. $$ Now assume that the formula holds for some $k$. Then, integrating by parts, $$\begin{eqnarray} \int_0^1p^{k+1}(1-p)^{n-(k+1)}\,dp&=&\int_0^1p^{k+1}(1-p)^{(n-1)-k}\,dp\\ &=&\left.-\frac{p^{k+1}(1-p)^{n-k}}{n-k}\right|_0^1+\int_0^1\frac{(k+1)p^k(1-p)^{n-k}}{n-k}\,dp\\ &=&\frac{(k+1)}{n-k} \frac{k!(n-k)!}{(n+1)!}\\ &=&\frac{(k+1)!(n-(k+1))!}{(n+1)!}. \end{eqnarray}$$ The induction principle then guarantees that the formula holds for all $k$.
This can be proven using repeated integration by parts: $$\begin{eqnarray} \int_0^1 p^k(1-p)^{n-k} &=& \frac{1^{k+1}(1-1)^{n-k}}{k+1}-\frac{0^{k+1}(1-0)^{n-k}}{k+1}+\frac{n-k}{k+1}\int_0^1 p^{k+1}(1-p)^{n-k-1}\\ &=& \frac{n-k}{k+1}\int_0^1 p^{k+1}(1-p)^{n-k-1}\\ &=& \frac{(n-k)(n-k-1)}{(k+1)(k+2)}\int_0^1 p^{k+2}(1-p)^{n-k-2}\\ &\vdots&\\ &=& \frac{k!(n-k)!}{n!}\int_0^1 p^{n}=\frac{k!(n-k)!}{n!}\frac{1}{n+1}=\frac{k!(n-k)!}{(n+1)!}\\ \end{eqnarray}$$
