Naive approach to Pythagoras The following has occupied me while learning about $a^2+b^2=c^2$, I then forgot about all that and recently (40yrs after) came across that again - and am still unable to understand. But today my next thought was to discuss the matter here and I'm sure we can quickly work it out! :-)
Unfortunately my graphic skills are worse than my math, so I'll have try to describe my idea textwise here, I hope my english skills will be good enough.
Let's assume we have a triangle with the angles $A$,$B$,$C$ ($A$ on top, $C$ being the $90°$ and $B$ being right) and the connecting lines being named with the lowercase name of the opposing angle. Sorry for not using proper terminology, I've been out of geometry for too long :((
The idea is that $c$ equals the distance of $A$ and $B$, and to determine that distance, we can (instead of the using the direct way $c$) also travel along $b$, then $a$ and get to $B$. So the distance is $a+b$.
Obviously we're going too far this way, so let's try to improve the route by "creating stairs". We go down b, but after half the way we turn right and walk hald the distance before turning again. So, this way the distance is ${a\over2}+{b\over2}+{a\over2}+{b\over2}$ or $a+b$.
And here's the point I do not get: following that idea, we can create an infinite number of steps, down to the width of an atom, which would approach the line of minimal length ($=c$) until they become one. But still: computing the length according to that approach, we'd end up with $a+b$ again! So, where is my fault???
 A: Each path with finitely many steps has a finite number of small wiggles, with the size of the latter being inversely proportional to the number. So in a very informal sense, the "limit path" has infinitely many infinitesimally small wiggles, such that the total amount of "wiggle" is positive but finite.
A: You have rediscovered one of the paradoxes of limits,
namely, that a sequence of paths in the plane can converge to a limit
path in the sense that the maximum distance to the limit path
from any point on your path converges to zero,
yet the lengths of the paths do not converge to the length of the limit path.
To put it simply, the detour through $C$ gives a longer path
than the direct path from $A$ to $B$; and taking many small detours can be
just as bad as taking one large detour.
There are many proofs of the Pythagorean Theorem if you're interested
(Pythagorean Theorem Proof Without Words 6 is an interesting example,
and there are several others under the question What is the most elegant proof of the Pythagorean theorem?),
but that seems to be a different question.
A: The reason why you keep on getting a + b as a result is that the formula that explains your procedure is:
Let n be the number of steps
$$ c = {\frac {a*n}{n}}+{\frac {b*n}{n}} $$
With n=2 we have your first example. The bigger n, the smaller the step, but the higher the amount of steps.
If it approaches infinite, we have your "infinite number of steps, down to the width of an atom" concept.
It does not matter how big n is, the end result will always be a+b.
To explain it conceptually, what you did was take a length, divide it in equal parts and then multiply it back for the number of parts it was divided, but that will always yield the original length as a result.
A: In the plane we have the notion of distance $d(X,Y)$ between any two points $X$ and $Y$. This distance is here since Euclid; it can be measured with a scaled ruler in a tenth of a second, and there is no infinitary procedure involved with such a measurement. 
The "paradox" you are describing just reveils that your process does not catch this notion of distance. But the following is true: When the two points $X$ and $Y$ are far apart, and your ruler is to short to measure $d(X,Y)$ in one move you can connect $X$ and $Y$ with a laser beam and then measure distances between subsequent intermediate points
$$X=P_0-P_1-P_2-\ldots -P_n=Y$$
along this beam. In the end the formula $$d(X,Y)=\sum_{k=1}^n d(P_{k-1},P_k)$$
is indeed true.
A: First, an explanation of why the diagonal is indeed the limit of the stairstep curves.  The idea with a limit of a sequence is that, given a certain amount of error you are willing to tolerate (call it $\varepsilon$), there is some index after which the things in the sequence are "good enough."  In this case, when there are $n$ stairsteps they remain within $\frac{a+b}{n}$ distance of the diagonal (I'm being conservative here), so whenever $n>N$ with $N=\frac{a+b}{\varepsilon}$, the error will be at most $\varepsilon$.  To reiterate, we say the diagonal is the limit of the sequence of stairsteps limits because they get arbitrarily close to the diagonal, where $\varepsilon$ parameterizes the closeness.  It seems you already intuitively understood this, but this is a somewhat more formal way of looking at it.
However, the limit isn't necessarily an element of a sequence itself.  For the diagonal, you can always get the stairsteps closer to the limit by increasing $n$ (which is proof that the diagonal isn't a bunch of stairs).
A simpler example to understand this might be $f(x)=\lim_{a\to\infty}e^{-ax^2}$ (link).  This function is zero for all $x$ except $f(0)=1$.  We might consider looking at the sequence $f(\frac{1}{1}),f(\frac{1}{2}),\dots,f(\frac{1}{n}),\dots$ and observing that it is $0,0,0,\dots$, then conclude "$f(\frac{1}{\infty})=0$" as well.  However $f(0)=1$.
Moral: the limit need not be like the others.
Extension: continuous functions are defined to be those functions where the limit is like the others.  When a function $f$ is continuous, $\lim_{x\to a}f(g(x))=f(\lim_{x\to a}g(x))$ (so long as the inner limit exists).  The stairstep sequence demonstrates that the arclength function does not satisfy this limit swap.
Elaboration: another way of measuring arclength is to, for each $n$, put down the minimum number of discs of diameter $\frac{1}{n}$ which cover the curve, add up the number of diameters of the discs it took to cover, then take the limit as $n$ approaches infinity.  If the arclength were indeed $a+b$, then it would take at least $n(a+b)$ discs of radius $\frac{1}{n}$ to cover the diagonal, but you can do much better than that.
