You have two specific questions here, one about exponentiation by repeated squares, and one about what branch of mathematics to do this in.
First, you want a proof that exponentiation can be performed by repeated squaring. You of course realize that exponentiation $a^b$ can be computed by multiplying $b$ copies of $a$. Another way to say this a la Sussman's SICP, is that:
$$a^b = a \times a^{b-1}, a^0 = 1$$
(this is just a recursive way of string on a list of $a$'s of length $b$).
The repeated squaring algorithm gets you exponentiation because (and I think this is the single important thing you're curious about) it also gets you a list of $a$'s of length $b$, just in a slightly more complicated way. The explanation is this: $b$ is either even or odd (pretty obvious right?). Let's take those two cases separately.
If $b$ is even, then $a^b = a^{b/2} \times a^{b/2}$, right? (because $b$ is even you can divide it by 2, and also, $a^x \times a^y = a^{x+y}$ a basic property of exponentials. (note that $b/2$ might be even or odd, we don't know).And this is the same as $a^{b/2}$ squared. That's where the repeated squaring comes from.
if $b$ is odd, then $a^b = a \times a^{b-1}$ (the usual formula). (note that $b-1$ must be even now).
Because $b$ is either even or odd (and never both) this covers all cases (there are no other possibilities than even or odd) and we can always get $a^b$ using some smaller exponents.
And since we are always getting smaller exponents, we always get down to an exponent of 1, no matter what the exponent is.
And we're done!
(why would we want to complicate matters this way? Well, we're (often) dividing by 2 so the number of multiplications is like the log of $b$ rather than $b-1$, which is a lot less.)
Now to your second question, what branch of mathematics is needed for this? The facts that we used were some properties of exponentiation, some number theory (properties of odd and even) and some arithmetic. These all tend to be taught, at least in the US education system in a first or second year of high school algebra.
There's one step of the proof that is more or less the glue logic that holds it all together and puts the last tiny jigsaw piece in to solve the puzzle and that was the last step, that you always get a smaller exponent and reach 1. The mathematics for that is induction, which is exactly what Sussman is teaching by this example. Induction is often taught in high school in various classes, maybe in algebra, maybe in precalculus.
A side note: if you know about binary, consider $b$ in binary. The repeated square algorithm is then following the binary digits backwards. If $b$ ends in a 0 (is even) then divide $b$ by 2 and square the result. If $b$ ends in 1 then change the 1 to a 0 (subtract 1) and multiply be one $a$. Then recursively work on the remaining digits.
(a^m)^n = a^(mn)
). In general, the problem of creating algorithms that compute known mathematical objects or functions is vast and it is usually addressed in the part of mathematics to which those objects or functions belong. $\endgroup$