This is indeed the idea. My understanding is you need to assume the Hilbert space in question is separable, though I'm not familiar with any counterexamples.
Suppose $\{e_{i}\}_{i \in \mathbb{N}}$ is a complete orthonormal system. For each $N \in \mathbb{N}$, consider the operator $T_{N}$ defined by
$$T_{N}(x) = \sum_{j = 1}^{N} \langle T(x), e_{j} \rangle e_{j}.$$
$T_{N}$ has range contained in $\text{span}\{e_{1},\dots,e_{N}\}$ so it's a finite rank operator.
Recall that $e_{i} \rightharpoonup 0$. Since $T$ is compact, it follows that $T(e_{i}) \to 0$ in norm.
We claim $\lim_{n \to \infty} \|T_{n} - T\| = 0$. Pick $\epsilon > 0$. By the preceding remark, we can find $\mathcal{I} \in \mathbb{N}$ such that $$\|T(e_{i})\|^{2} < \epsilon^{2}$$
if $i \geq \mathcal{I}$. Moreover, since $\{1,2,\dots,\mathcal{I} - 1\}$ is finite, we can find $N \in \mathbb{N}$ such that
$$\sum_{j = n + 1}^{\infty} |\langle T(e_{i}), e_{j} \rangle|^{2} < \epsilon^{2}$$
if $i \in \{1,2,\dots,\mathcal{I}\}$ and $n \geq N$.
Now suppose $x$ is any vector in the Hilbert space. If $n \geq N$, then
\begin{align*}
\|(T - T_{n})(x)\|^{2} &= \sum_{j = n + 1}^{\infty} |\langle T(x), e_{j} \rangle|^{2} \\
&= \sum_{i = 1}^{\infty} |\langle x, e_{i} \rangle|^{2} \left(\sum_{j = n + 1}^{\infty} |\langle T(e_{i}),e_{j}\rangle|^{2}\right) \\
&\leq \sum_{i = 1}^{\mathcal{I} - 1} |\langle x,e_{i} \rangle|^{2} \left(\sum_{j = n + 1}^{\infty} |\langle T(e_{i}),e_{j} \rangle|^{2}\right) + \sum_{i = \mathcal{I}}^{\infty} |\langle x,e_{i} \rangle|^{2} \|T(e_{i})\|^{2} \\
&\leq \epsilon^{2} \sum_{i = 1}^{\infty} |\langle x,e_{i} \rangle|^{2}
\end{align*}
by the choice of $\mathcal{I}$ and $N$. From this, we deduce that $\|T - T_{n}\| \leq \epsilon$. Since $\epsilon$ was chosen arbitrarily, we conclude $\lim_{n \to \infty} \|T - T_{n}\| = 0$.