I have rewritten this entire question, since what I've learned since asking it requires me to restate it. I want to get rid of the obfuscating revisions.
Let's say that f is a continuous function.
$f(x)$ approaches L as x approaches a. So $\lim\limits_{x \to a}f(x) = L$
When it's said that the gradient of a tangent line to a curve at some particular point has some particular value, this is the same as saying f(a)=L. But without explicitly evaluating at that point, you can't say as much. All you can say is what happens as you approach that value. In other words, you can only say what value $f(x)$ approaches as $x$ approaches $a$, you can't say what $f(a)$ is.
Is that true?
Some textbooks will just say that the value of $f(a)=L$. Sal Khan's explanation does this. He says, about the function as it approaches the limit, "this is the gradient of the tangent." I'm saying that it should be said that, "the derivative approaches the gradient of the tangent to the curve." I think "approaches" and "is" are very different.
If there is a way to prove that the $f(a)=L$ then I'd like to see it. I don't know how to do this yet.