A calculus exam is designed to test whether you know calculus, not its prerequisites. Your professor probably assumes you are well-versed enough in arithmetic and algebra that you will not make substantial "simple" errors. If you know the calculus,[*] then, but you're consistently making errors in arithmetic and algebra on the exam, then I'd suggest you need to change the focus of your studying.
Try digging up an old arithmetic or algebra textbook and working through the exercises. Do 10 or 20 at a time, then stop and grade yourself on accuracy. Keep doing this until you consistently score perfectly on these mini-exams. You're essentially drilling these skills into your subconscious so that you intuitively know how these manipulations work, and if you make a mistake, you immediately catch it.
When you're studying calculus proper, you probably go over old exam problems and homework exercises and do them. When you check your solutions to these and run across arithmetic errors, don't assume they were "just" silly mistakes and move on. You've identified that your problem on exams is exactly this, so you need to figure out where you're going wrong with the algebra/arithmetic in your problem-solving process and start working to correct it.
Here's a sports analogy. Imagine you're on a basketball team and you've got a great grasp of strategy and court sense --- you know where your teammates are, where the defenders are, who's open, and who's not. However, when you have the ball, you can't seem to control it. Your dribbling is wild and when you pass or shoot, it doesn't go where you thought you put it. In practice, what do you do? You spend hours just dribbling and then you line up and take a thousand shots at the basket. When you're playing scrimmage and the ball slips out of your hands, do you ignore it? No, you think about why you lost control and how to do better the next time. In the same way, you need to drill yourself on the prerequisites for calculus and be more self-critical when you don't understand the algebra in a problem.
[*] By this I mean that when you see a problem, you know very quickly how to solve it. For instance, if you see "Find the area between $f$ and $g$," you know immediately that you need to find the intersection points of $f$ and $g$ and the integrate $|f-g|$ between those intersection points.