I'm not sure if you are still looking for clarification on this issue, but here's a go.
For simplicity, let's assume $y=f(x)$ is differentiable for all $x$, and $(x_0, y_0)$ is a point on the curve defined by this equation. (I'll use $x_0$ and $y_0$ instead of $a$ and $b$, for convenience in an explanation below.) The derivative is defined so that $f'(x_0)$ is tangent slope to the curve $y=f(x)$ at the point $(x_0, y_0)$. Then, indeed point-slope form gives us the equation
$$ y - y_0 = f'(x_0)(x-x_0). $$
Note what you wrote was not quite right: $y-b = f'(x)(x-a)$ is not true for all $x$, only in general when $x=a$ (so $y=b$ and both sides will be $0$). This was probably the source of your confusion. I believe what you were thinking about is trying to reconstruct the curve $y=f(x)$ from knowing the derivative $f'(x)$ and a specific point $(a,b)$. This is an important topic actually, and is the basic premise behind antiderivatives and differential equations.
Put another way, the equation for the tangent line has 3 components that depend on the specific point: $x_0$, $y_0$ and the slope $f'(x_0)$ at that point, so you can't just replace the $f'(x_0)$ with an arbitrary $f'(x)$ to get the curve, but keep $x_0$ and $y_0$ fixed. (By the way, the curve you got by doing this faulty procedure is not the tangent line either because it's not linear--I don't know that that curve has much meaning.)
Here's a better way to think about trying to reconstruct the curve from the derivative and a specific point. I'll try to explain this as briefly as I can, but feel free to ask for clarification.
Let's work with the simple example of $y=f(x) = x^2$, and pick the starting point $(x_0, y_0) = (0, 0)$. Here $f'(x)= 2x$,
so at the starting point the slope is $m_0 = f'(x_0) = 0$. This means the curve is close to the tangent line $y-y_0 = m_0(x-x_0)$, i.e., $y=0$ nearby the point $(0,0)$. That is, we can approximate the curve near $(0,0)$ geometrically by drawing a short horizontal tangent through $(0,0)$.
Now we can do this again. For simplicity, let's say we approximate our curve with the horizontal tangent to $(x_1, y_1) = (1, 0)$ (this is not a point on the curve, but on the tangent line). Now we can use the derivative at 1, $f'(x_1) = f'(1) = 2$, to draw another line segment, say to $(x_2,y_2) = (2,4)$, which is an approximation of the tangent line to $y=f(x)$ at $x_1$.
This is much easier to understand with pictures--take a look at the pictures here for what's going on. I'm just saying we can successively use derivatives (tangent slopes) to approximate the curve. Of course my approximations in my example were very bad, but they get better and better by taking smaller steps (e.g., $x_1 = 0.1$ or $0.00001$, and $x_2 = 2x_1$).
To actually recover the curve $y=f(x)$ exactly, you need to take infinitely small steps, or more precisely, take the limit of doing this process with smaller and smaller steps. This is precisely what the antiderivative is.