Questions tagged [computer-arithmetic]

For questions concerning finite precision arithmetic in computers and other related concepts.

29 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
108 views

Mathematics of two's complement

I am trying to understand the underlying mathematics of two's complement. Googling the topic gives me a lot of articles on how to invert the digits and add one, and why computers use this system ...
3
votes
0answers
143 views

Error bound for floating-point interval dot product

In Handbook of Floating-Point Arithmetic (Birkhäuser, 2010, Chapter 6) Muller et al. presented the following absolute forward error bound for the floating-point recursive dot product: $$ \left|...
2
votes
0answers
30 views

Is there any way to predict the amount of “carries” one could get by doing a sum?

https://en.m.wikipedia.org/wiki/Carry_(arithmetic) Definition of carry in Wikipedia I'm guessing the answer is no but just to be sure I wanted to know of there is a way to predict the amount of ...
2
votes
0answers
82 views

SICP: Why does this recursion-based sine approximation work?

Here is the question and solution to Structure and Interpretation of Computer Programs' exercise 1.15 (see here). My problem is, I don't know how the combination of these formulae actually work: $$...
2
votes
1answer
445 views

How would you reduce roundoff error in “mod” when implementing a periodic function?

I recently wrote How calculators do trigonometry where I wrote a simple program for computing $\sin(x)$. For completeness, I include a slightly modified version of the program here: ...
2
votes
1answer
146 views

Numerical error analysis

I stand before the following task and I do not know how to solve it. The input parameter $$a=10^6, b=10^6 + 10^{-2}$$ will be round internally to $$a^*, b^*$$ with $$a=a^*(1 + \epsilon_1)$$ $$b=b^*(...
1
vote
0answers
20 views

Resultant of multivariate polynomials

I know little about resultant. As for as I know, if two univariate polynomials $f,g$ have a common zero iff $Res(f,g)=0$, is it right? Now I see an example using resultant of two multivariate ...
1
vote
0answers
36 views

Multiplication of medium sized integers

Implementations of integer multiplication often choose different algorithms for different input sizes: for smallest numbers the school book method is preferred, for largest sizes the method based on ...
1
vote
0answers
268 views

Solving BCD using 10s complement

25-7 is the question. (0010 0101 - 0111). Change 0111 to 0011 (using 10s complement) and add it, isn't it? I've been trying to solve using 10s complement but couldn't. Please help me solve the ...
1
vote
0answers
34 views

Error analysis of a non restoring division algorithm. Studying the iteration.

I'm not sure my derivation is correct, and I also need to find out the error of the iteration/sequence I'm about to derive, but I can't figure out the error once the iteration is finished. Let $x,y,q,...
1
vote
0answers
68 views

Coding for Mathematics

I don't think this is the appropriate group to ask but since most of the people dealing with mathematics are involved, I prefer to ask here instead of the Stack Overflow site. For my math works, it ...
1
vote
0answers
76 views

Proof that the Booth encoding and the modified booth encoding are correct

In this book, sections 9.4 and 10.2 explains how to implement the booth encoding to speed up operations in hardware. I've being searching for some reference for a formal proof of why this encoding is ...
1
vote
0answers
615 views

Hanek payne trigonometric range reduction detailed analysis of loss of significance.

I'm struggling to understand what is the actual problem described as "loss of significance" when a trigonometric range reduction argument is performed. I was trying to perform the analysis by myself ...
0
votes
0answers
4 views

How to generate Maximum Min-SW gray code?

I'm trying to generate Maximum min-SW gray code for structured light described in this paper. Can someone help to understand how to generate this 10 bit gray code?
0
votes
0answers
18 views

Identify all the RAW data dependencies in the following code.

Identify all the RAW data dependencies in the following code. Which dependencies are data hazards that will be resolved by forwarding? Which dependencies are data hazards that will cause a stall? ...
0
votes
0answers
21 views

ASCII code. Binary representation

A character representing the digit n is stored as an 8 bit ASCII code. Convert to an 8 bit sign magnitude for the negative numerical value,-n. What is the algorithm for doing this?
0
votes
0answers
15 views

IEEE 754 Exponent range

It is about the normalized floating-point representation of real numbers (IEEE standard 754). Why does the data type double (64 bits) with mantissa length with 53 bits for the exponent range [-1022, ...
0
votes
0answers
54 views

Required Precision in Calculating Reciprocals of $n$ and $\frac{1}{n}$ to Recover the Integer

Suppose I am given a large integer $n$ and a small positive value $\epsilon$ such that I am to compute $\frac{1}{n}$, and then compute the reciprocal of that, in order to recover $n$ to within an ...
0
votes
1answer
17 views

Relative roundoff error in the simple precision.

I am struggling with a simple problem from the computer arithmetics. The goal is to find $|(fl(0.1)-0.1)/0.1|$. I have computed the binary representation of $0.1$, which is $(1.1001)_{2}\times2^{-4}$, ...
0
votes
0answers
60 views

Tens Compliment Arithmetic.

For the following pairs of 4 digit decimal numbers, A and B, show the ten's complement of each, compute A + B and A – B using ten's complement arithmetic, indicate which of the 4 numbers (A, B, A + B, ...
0
votes
0answers
85 views

What is the maximum number of significant bits lost when the computer evaluates x − y using IEEE 64 bits?

Consider two positive numbers $x = p2^m$ and $y = q2^n$ such that $m > n$, $1 < p < 2$, and $1 < q < 2$. Both of these numbers can be stored using the IEEE 64 bit standard. What is ...
0
votes
1answer
28 views

Adder delay model in a ripple carry adder.

In section 5.3. of the following book an analysis of carry propagation in the ripple carry adder is performed. However the statistical analysis doesn't particularly convince me. Specifically at the ...
0
votes
0answers
140 views

Rearrangement algorithm

I came across the following algorithm puzzle while interviewing for computer programmer job with a top internet giant. There are $n=4$ bags with each bag able to carry a maximum of $k=500$ Kilogram ...
0
votes
0answers
34 views

Proving correctness for code computing function.

I was solving an exercise from the book "Complexity, computability and languages" which asks: Write a program that computes $f(x)=1 \iff x$ is even, $f(x)=0\iff x$ is odd. I wrote the following ...
0
votes
1answer
17 views

Computer arithmetic - size of error.

I'm actually reading What Every Computer Scientist Should Know About Floating-Points Arithmetics and I have a little problem with understaing one thing. Here's a quote: Write $(b \otimes b) \...
0
votes
1answer
45 views

standard notation to handle representation of a real number on a computer

Is there a standard notation to handle the effective representation of some real number $x$ on a finite machine ? I have in mind some kind of braces, but I am not sure it is appropriate. Let me try to ...
0
votes
1answer
56 views

Making a function for unbounded variables that is bounded

Ok here is what I need I want to build a formula to implement into a ranking system on my website its been like 20 years since I was in school so help me out I need like 6 variables that equal at ...
0
votes
1answer
99 views

Understand a Maple output

My goal is to solve for $L$ in $\frac{(2k)!}{2^kk!}{2nL - L^2 \choose 2k} = \sum_{s=0}^k{L \choose s}{n-L \choose s}s!\frac{(2k-2s)!}{2^{k-s}(k-s)!)}{L-s \choose 2k-2s}.$ I tried to use the solve ...
-1
votes
1answer
17 views

Asking about a union of languages and how they interact

if we take a language, $L= \{b^{2n}a^n \mid n \gt 0\} \cup \{b^{3n}a^n \mid n \gt 0\}$ would this be the same as $L=\{b^{2n}a^n b^{3n} a^n \mid n \gt 0\}$ ?