# Calculator - Why can I not calculate 7F FF FF FE * 2

I just tried to calculate on my CASIO fx-991 ES the following equation:

$$7F FF FF FE_{16} \cdot 2$$ and $$7F FF FF FE_{16} + 7F FF FF FE_{16}$$

My result was MATH ERROR.

I therefore expected I overshot the 32 bit limit of the base mode of my calculator. I concluded that if that happened in a processor, the "carry flag" would be set and continued to compare my result with given solution. I was wrong. After trying to figure out why and calculating this by hand, I realised the solution is $$FF FF FF FC_{16}$$. When I type this directly into my calculator it accepts the input without hesitation.

Does anyone have any idea what kind of math error I ran into if the result is clearly not too big for calculation? (By the way, if I take one digit away, it's fine with the calculation. But still, why does this happen before violating the max length restriction?)

Oh, and a final thought. This is apparently calculator specific. The windows calculator has no issue with the above input.

• A wild guess. This was interpreted as a signed integer, when the range may be from $-80000000_{16}$ to $7FFFFFFF_{16}$. With the latter input the calculator was forced to take it as unsigned. Commented Jul 26, 2022 at 18:29
• You're right. $7FFFFFFF_{16} + 1$ also produces a math error. But $FFFFFFFE_{16} + 1$ does not produce a math error. That's exactly it! Hey, can you make it a real answer so I can give you the best answer? Commented Jul 26, 2022 at 18:32
• You say you typed in $FFFFFFFC_{16}$ and it was accepted without hesitation. If you convert it to decimal it should show $-4$, something else to be aware of when working with signed integers. Commented Jul 26, 2022 at 18:50

I don't know the inner workings of this particular calculator. But my wild guess theory is that your inputs were interpreted as a signed integers. Often the range of a signed integer is from $$−80000000_{16}$$ to $$7FFFFFFF_{16}$$, and this overflowed. With the latter input the calculator was forced to take it as unsigned.