Let's look at the the Lie algebra $\mathfrak{sl}(2, \mathbb{C})$ with basis $E, F, H$ as in your example. (So $H$ generates the Cartan, $E$ and $F$ are 'step operators' satisfying $[H, E] = 2E$, $[H, F] = 2F$, $[E, F] = H$.)
One construction you have probably seen (given your post) is to start with a lowest weight vector $v \in V$ which is an eigenvector of $H$ (i.e. $\rho(H)v = \lambda v$) and which satisfies $\rho(F)v = 0$.
Now, (more or less as you write) you get different vectors in $V$ by applying $\rho(E)$. We obtain $\rho(E)v$ which is an eigenvector of $H$ with eigenvalue $\lambda + 2$, we get $\rho(E)\rho(E)v$, which is an eigenvector of $H$ with eigenvalue $\lambda + 4$, we get $\rho(E)\rho(E)\rho(E)v$ etc etc etc.
Since the vectors obtained in this way are eigenvectors for $H$ at different eigenvalues, they are linearly independent. Continuing in this way we get, it seems, an infinite set of linear independent vectors in $V$, so naturally $V$ is infinite dimensional (and non-trivial, e.g. irreducible).
So one thing you have to ask yourself is: how come all examples I saw are finite dimensional instead?
This happens when at some point $\rho(E)$ acts as the zero operator.
So we reduce the question to 'when does this happen'?
To understand this look at the action of the Casimir operator $\omega = H^2 + EF + FE$. It has the nice property that it commutes with everything in $U(\mathbb{g})$.
Using this, you can check that the eigenvectors for $\rho(H)$ you created by successive application of $\rho(E)$ are also eigenvectors for $\rho(\omega)$ and even for the operator $\rho(FE)$. If this latter thing equals zero on some eigenvector, that forces some conditions on the corresponding eigenvalue for $H$. (EDIT: what I forgot to type 7 years ago is where these conditions come from. The point is that while all these eigenvectors each have their own eigenvalue when considered an eigenvector of $\phi(H)$, the all have the same eigenvalue when considered as an eigenvalue of $\rho(\omega)$. This is where you use that $\omega$ commutes with everything. END OF EDIT)
In the end the answer turns out to be that if the vector $v$ you started with had $H$-eigenvalue $\lambda = -n$ for some positive integer $n$ then $\rho(E)$ acts as zero on the eigenvector with eigenvalue $+n$, and we get an $n$-dimensional representation. (That is: the subrep of $V$ generated by $v$ is $n$-dimensional). But for all other values of $\lambda$ (e.g. non-integral or positive) the subrepresentation generated by $v$ is infinite dimensional since $\rho(E)$ is injective on it. (And hence $V$ must have been infinite dimensional from the start as well.)
See 'Verma module' for more information on this construction.
So put this way, the finite dimensional ones are really the exception to the rule!
But...
If you have a compact Lie group then all irreducible representations are finite dimensional. Conversely this means that not all irreducible representations of a Lie algebra come from representations of the compact real group associated to it. (Some infinite dimensional ones do come from group reps of non-compact groups and some don't integrate to group reps at all.)