Is there really such a thing as a tangent line? Is there really such a thing as a tangent line? Even in the definition of a derivative, you always get:
$$\frac{d}{dx} f(x)= \lim_{h\to 0} (f'(x)+h)$$
Although that h approaches 0, it can never be 0. Think of it this way. If I show you a picture of somebody running and ask you "at what speed is he/she running" you would never be able to tell me. I would have to show you 2 pictures of that runner and tell you the amount of time between the first and the second picture in order for you to tell me the speed. So does a true tangent line really exist? (except maybe in the case of a circle).
 A: Going with your analogy, first note that in general we define $f'(x) = \lim_{h\rightarrow 0} \frac{f(x+h)-f(x)}{h}$. Actually this contains more "information" compared to your picture example (You have more than $2$ pictures, in fact you have infinitely many pictures). First, you have one picture of the runner, which is $f$ evaluated at $x$, $f(x)$. You also take into account many pictures of the runner after infinitesimal time intervals after the moment $x$. These pictures are precisely $f(x+h)$ as $h\rightarrow 0$. Making sure that $h\neq 0$ actually helps here, so that you always have more than one picture of your runner.
A: Here is one way to think about tangent lines.
Let $C$ be any collection of points in the plane.  For any point $P\in C$, we say that a line $L$ is quasi-tangent to $C$ at $P$ if for some open ball $U$ of $P$, the following two conditions hold:


*

*$U\cap L\cap C=\{P\}$ or $U\cap L\cap C=U\cap L$

*If $U_1$ and $U_2$ are the two connected components of $U\setminus L$, then at least one of $U_1\cap C$ and $U_2\cap C$ are empty.


Notice that there may be infinitely many $L$ that are quasi-tangent to $C$ at $P$.  For example, if $C$ is the graph of $y=|x|$, then the point $(0,0)\in C$ has infinitely many quasi-tangent lines.
Now suppose that only one such $L$ exists.  Then we say that $L$ is tangent to $C$ at $P$, and we can define the slope of $C$ at $P$ to be that of $L$.  So far, we haven't explicitly referenced any limit (although the limit is probably hiding somewhere).
Intuitively, it seems that this definition should be equivalent to the usual definition when $C$ is the graph of some smooth function (see the comment below for a counterexample when the function is differentiable, but not smooth).
A: Part of the problem is you're trying to compare mathematics to reality. As a mathematical object of course tangent lines exist, but the question of to what degree real world phenomena may be accurately modeled by continuous/differentiable functions is another. No one even knows if spacetime is discrete, continuous, non-commutative etc. You could also ask the question "Do spheres exist?", and the answer again would depend on whether you mean do they exist as mathematical objects or do they exist in nature. I would answer "yes" to the former and "no" to the latter, but that's just me.   
A: Tangent lines exist as geometric objects and they have slopes. The real question is why does the derivative equal the slope of the tangent line? The answer is that the derivative $f^{'}(x)$ is not the slope of a secant line for a point really close to $x$. It's the limit of the slopes of the secant lines for points arbitrarily close to $x$.
If I say $\lim_{x\to c} f(x) = L$ I don't mean $f(c) = L$. I mean that if you specify any tolerance on L, I can get within that tolerance by choosing a value for $x$ closer to $c$.
A: I'm thinking of a number.
I'm not going to tell you what my number is. But I will tell you that $1$ is close to my number. And that $-0.1$ is closer to my number. And that $0.01$ is even closer to my number. And $-0.001$ is even closer still.
Of course, that's not enough information to figure out my number. I will additionally tell you that each of the numbers $(-1)^n 10^{-n}$ (where $n$ ranges over nonnegative integers) get successively closer to my number; each of these number is closer to my number than the one before it.
Surely from this information, you can figure out what my number is. Despite the fact I never told you my number, and none of the numbers in the sequence I mention is equal to my number.
That is what a limit does.
A: Thinking about the derivative at a point as the slope of the tangent line at that point is simply a way to guide one's intuition via geometry.  Further, it is not true that the derivative (or tangent line) always exists. For example, the absolute value function $y =|x| $ does not have a well defined tangent line at $ x = 0$
Lastly, yes in practice, the derivative is often estimated using similar methods as you've stated, but even in the example you gave, one can argue that the speed of the runner at a particular instance exists, so the derivative's existence is not the issue -- the problem is the derivative is hard or impossible to give precisely in some real-world situations, but it can usually be estimated if it exists.
But I should also say, the 'existence' of a mathematical idea in the real world is really a matter of philosophy. 
