Calculus is all about the interplay between differentiation and integration. Looking at the definition of the derivative, you'll see that it only really uses two properties of the real line: its linear structure (so that we can form the difference $f(x + h) - f(x)$ and its topological structure (so that we can take a limit). So in this sense, calculus is exactly half algebra.
Of course, that doesn't explain why it feels like mostly algebra when you're actually taking a calculus class. When you get right down to it what you use on a day-to-day basis in a calculus class is the relationship between differentiation and integration, and this relationship can be abstracted as follows. Begin with the ring $A$ of all smooth functions on $C^\infty(\mathbb{R})$. We have two linear maps $d$ (differentiation) and $I$ (integration) on $A$ which satisfy the following properties:
- $d(1) = 0$
- $d(f \cdot g) = df \cdot g + f \cdot dg$
- $d(f \circ g) = dg \cdot df \circ g$
- $d \circ I = id$
- $I \circ d = id + C$
Taking these five properties as axioms, you can differentiate and integrate basically any function that you encounter in your first year of calculus. For instance, the second and third give you the quotient rule and the second and fifth give you integration by parts. In fact, with a bit of extra effort one can embed these axioms into an algebraic package which completely characterizes $C^\infty(\mathbb{R})$ and hence most of calculus.
A final remark. Notice that in your first year of calculus you don't actually encounter all that many functions. Here is a nearly exhaustive list:
- Polynomials
- Exponentials
- Products, quotients, compositions, and inverses of the above
You may object that I forgot trigonometric functions, but thanks to de Moivre's equation $e^{ix} = cos(x) + i sin(x)$ these are really just exponentials. Polynomials are certainly algebraic objects, and exponential functions can be characterized as the only group homomorphisms from the additive group of the real line to the multiplicative group of the ray $(0,\infty)$. So it is not unreasonable to expect to do a lot of algebra given that you're only working with algebraic objects.