True for all $n$ implies true as $n$ tends to $\infty$?

I'm doing some exercises and came across one that has two parts, as follows:

Given a transition matrix for a Markov Chain, $\mathbf{P}$, and a vector $\mathbf{f}$, $\mathbf{f}$ is harmonic if

$$\mathbf{f} = \mathbf{P}\mathbf{f}$$

$(a)$ Show that if $\mathbf{f}$ is harmonic, then

$$\mathbf{f}=\mathbf{P}^n\mathbf{f}$$

for all $n$

$(b)$ Using $(a)$, show that if $\mathbf{f}$ is harmonic,

$$\mathbf{f} = \mathbf{P}^\infty \mathbf{f}$$

Am I incorrect in assuming that if $(a)$ holds, then $(b)$ holds by necessity? Are there any cases where proving that something holds for all $n$ does not prove that it holds as $n$ tends to infinity?

-
You need to refer back to the definition of $\textbf{P}^{\infty}$. – Qiaochu Yuan Sep 6 '12 at 19:38
If something holds for all $n$, then it clearly holds for $n$ as $n$ tends to $+\infty$. But you don't seem to be asking what happens as $n$ tends to $+\infty$, but instead what happens at $+\infty$. – Hurkyl Sep 6 '12 at 20:22
@Hurkyl "If something holds for all $n$, then it clearly holds for $n$ as $n$ tends to $+∞$." What about the counterexample below? – Pedro Tamaroff Sep 7 '12 at 1:47
@Peter: It's an example of a statement that holds for every term in any sequence of natural numbers tending towards $+\infty$, but whose (suitably interpreted) limit fails. – Hurkyl Sep 7 '12 at 4:10

Simple counterexample

$$\frac{1}{n}>0\text{ for all }n\in\Bbb N$$

-

The sum $$\sum_{k = 1}^n \frac{1}{k}$$ is finite for all finite $n$, but $$\sum_{k = 1}^\infty \frac{1}{k}$$

is infinite.

-
OK, let me be a nitpicker here, but what does it mean for a finite sum to be convergent? – Pedro Tamaroff Dec 3 '12 at 22:59
@PeterTamaroff I agree it is an abuse of terminology. I meant only that the sum of finitely-many finite terms is finite. I've replaced my answer with something more concrete. – Austin Mohr Dec 4 '12 at 0:37