# Questions tagged [computer-arithmetic]

For questions concerning finite precision arithmetic in computers and other related concepts.

80 questions
Filter by
Sorted by
Tagged with
27 views

77 views

### Proof that the Booth encoding and the modified booth encoding are correct

In this book, sections 9.4 and 10.2 explains how to implement the booth encoding to speed up operations in hardware. I've being searching for some reference for a formal proof of why this encoding is ...
56 views

### Building a sequence that approximates given sequences

Suppose that we are given three sequences $a1,a2$ and $a3$ each describing a total ordering on $N$ 'entities'. For example, $$\langle a1\rangle=1<9<8<2<3<\cdots<N$$ means that ...
36 views

### Multiplication of medium sized integers

Implementations of integer multiplication often choose different algorithms for different input sizes: for smallest numbers the school book method is preferred, for largest sizes the method based on ...
64 views

94 views

### Conjectures that are false for very large numbers [duplicate]

Hi I would like to teach my students about false conjectures and computer approaches to them. I need references and/or direct examples of statements of the form $(\forall n \geq n_0)P (n)$ where $P$ ...
132 views

### Converting $\frac{2}{7}$ to a binary number in a $32$ bit computer

I want to convert $\frac{2}{7}$ to a binary number in a $32$ bit computer. That is, $1$ bit is assigned to the sign of the number, $8$ bits are assigned to the exponent, and $23$ bits are assigned to ...
39 views

### Computing $x_+ - x$ where $x_+$ is the rounded binary number of $x$

This question stems directly from Converting $\frac{2}{7}$ to a binary number in a $32$ bit computer I want to check if $x_+$ (increase the value of the last bit by one unit and discard all the bits ...
270 views

### Why do so many computer programming language implementations have trouble with the remainders of negative integers?

As most of us know, or should know, $-7 \equiv 1 \pmod 4$. But if you use Java's modulus operator %, you get -3 for the answer, ...
108 views

### Mathematics of two's complement

I am trying to understand the underlying mathematics of two's complement. Googling the topic gives me a lot of articles on how to invert the digits and add one, and why computers use this system ...
20k views

### Arithmetic Overflow and Underflowing

I am a bit unclear about underflowing in terms of binary representation. Let's say that an unsigned 8-bit variable gets overflown from the addition of $150+150$. A signed 8-bit variable gets ...
268 views

### Solving BCD using 10s complement

25-7 is the question. (0010 0101 - 0111). Change 0111 to 0011 (using 10s complement) and add it, isn't it? I've been trying to solve using 10s complement but couldn't. Please help me solve the ...