In Serre's paper on $p$-adic modular forms, he gives the example (in the case of $p = 2,3,5$) of $\frac{1}{Q}$ and $\frac{1}{j}$ as $p$-adic modular forms, where $Q = E_4 = 1 + 540\sum \sigma_{3}(n)q^n$ is the normalized Eisenstein series of weight 4 and $j = \frac{\Delta}{Q^3}$ is the $j$-invariant.

To see this, Serre remarks that the fact that $\frac{1}{Q}$ is a $p$-adic modular form follows from the observations that $\displaystyle\frac{1}{Q} = \lim_{m\to\infty} Q^{p^m} - 1$ and that $Q = 1 \mod p$. He remarks that similarly $\frac{1}{j}$ can similarly be shown to be $p$-adic modular weight 0 and that the space of weight 0 modular forms is precisely $\mathbb{Q}_p\langle \frac{1}{j} \rangle$.

Forgive my ignorance, but could someone explain these facts in detail? Perhaps I am missing something obvious, but I don't understand why the $\displaystyle \lim_{m\to\infty} Q^{p^m} - 1 = \frac{1}{Q}$ is true and how to obtain the other statements he makes.

-

I'm not sure this can quite be correct. The problem is that $Q^{p^m}$ is going to tend to 1, so $Q^{p^m} - 1$ tends to 0, not $1/Q$. I think you may have misread the paper and what was meant was $1/Q = \lim_{m \to \infty} Q^{(p^m - 1)}$; if you're reading Serre's paper in the Antwerp volumes, then this is an easy mistake to make given that it's all typewritten! It shouldn't be too hard to convince yourself by induction that $Q^{p^m} = 1 \bmod p^{m+1}$.
As for $1/j$, we have $1/j = \Delta / Q^3$ (there is a typo in your post, you write $j = \Delta / Q^3$ which is not quite right) so once you know that $1/Q$ is $p$-adic modular it follows immediately that $1/j$ is so.