The whole point is that the limit of the difference quotient is the definition of the derivative at a point. In general, there is no independent standard against which to check this definition. As a result, it does not make sense to ask "does the value of the limit really give the slope of the tangent line?" -- precisely because we have no other definition of "slope of the tangent line," in general. Put more informally, there can be no question of whether the limit is "exactly" equal to something else, because we have no definition of that something else.
However, perhaps you do think you have an independent definition of "tangent line." If so, you probably have in mind the intuitive idea that a tangent line ought to intersect the curve "at one point." Of course this is wrong in general (think about tangent lines to $y=x^3$ away from $x=0$), but you can check that the limit definition gives the "right" answer (in this sense) for the simplest possible case of $y=x^2$ by a standard precalculus argument. Suppose the line $y=ax+b$ intersects the parabola $y=x^2$ at exactly one point, $(x_0,y_0)$. Then the equation
$$x^2-ax-b=0$$
ought to have precisely one solution (namely $x_0$). But the solutions are $x=\frac{a\pm\sqrt{a^2+4b}}{2}$. Since we have only one solution, the discriminant vanishes, giving $x_0=\frac{a}{2}$. In other words, $a=2x_0$, so the slope of the tangent line $y=ax+b$ is indeed the same as the limit of the difference quotient.
The problem is that this simple geometric picture won't carry you very far: it fails, as I said, even for the nice curve $y=x^3$. (Things can get a whole lot worse: consider the problem of making sense of the tangent line to the curve $y=x^2\sin(1/x)$ at $x=0$.) So we must take a different approach to defining "slope of the tangent line" for general functions. The approach we take is to say a function has a tangent line if the limit of the secant slopes exists.
Why is this a good approach? The short answer is that it allows us to prove many important theorems. But note, as well, that it does generalize some of our geometric intuition, which says that the secant lines ought to slide continuously through the tangent line as $h$ changes from positive to negative. In other words, the limit definition tells us how to define the function $$g(h):=\begin{cases}\frac{f(x+h)-f(x)}{h}& h\neq0\\??&h=0\end{cases}$$
if we want $g$ to be continuous at zero.