Another Solution could be as follows:
It's a double counting of the surjective functions from the set $\{1,\dots,j\}$ to the set $\{1,\dots, p\}$.
On one hand, clearly these functions are $p!$ if $j=p$, while there are no such functions if $j<p$.
Let's count these functions in a different way:
For $1\leq i\leq p$ let us define the sets $$A_i:=\{\text{the functions from }\{1,\dots,j\}\text{ to }\{1,\dots,p\}\text{ which doesn't contain }\; i\;\text{ in their image}\}.$$
We have then
$$\begin{align}
|A_i|&=(p-1)^j\\
|A_i\cap A_l|&=(p-2)^j\\
&\vdots&\\
\left|\bigcap_{i=1}^p A_i\right|&=0.\end{align}$$
We are interested in calculate the following: $$|\overline{A_1}\cap\overline{A_2}\cap\dots\cap\overline{A_p}|=|\overline{A_1\cup A_2 \cup\dots\cup A_p}|= p^j-|A_1\cup A_2 \cup\dots\cup A_p|=
$$
$$p^j-\sum_{i=1}^p\left((-1)^{i+1}\sum_{1\leq j_1<\dots<j_i\leq p}\left|\bigcap_{k=1}^i A_{j_k}\right|\right)=\sum_{k=0}^p(-1)^{p+k}\binom{p}{k}k^j.$$ From which the thesis follows.
EDIT Since I spent a lot of time on this problem a year ago, I would like to give two other pennies on the topic, and I would give another solution, or at least a sketch of it:
So, let us consider the differential operator which realizes: $$D(f(x)) = \frac{\mathrm d}{\mathrm dx}(x\cdot f(x)) = f(x) + x\cdot\frac{\mathrm df}{\mathrm dx}.$$ Immediately one sees that $$D(x^m)=m\cdot x^m,$$ hence $$\sum_{k=0}^{p}(-1)^k \binom{p}{k}\,k^j = \left.D^j\left((1-x)^p\right)\right|_{x=1}.$$ So, (almost) immediately, again the thesis follows.
Let me say one last thing. This formula is strictly related to Stirling numbers of the second kind, but note how the second proof I gave you does not say much on what happens if $j>p$, while the first fits perfectly even in this situation.